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ABSTRACT 


This  report,  summarizes  tin*  work  done  by  the  Information  and  Control 
Laboratory  during  19<>7  in  the  area  of  optimal  guidance  and  control  in  a 
missile  defense  system.  General  problem  formulations  for  the  optimum 
interception  of  both  ballistic  and  maneuvering  reentry  vehicles  are 
given.  A  computer  program  based  on  dynamic  programming  for  prelaunch 
calculations  is  described.  A  second  computer  program  that  utilizes  the 
giadii'iit  method  loi  in  Might  guidance  is  also  discussed,  filially,  a 
game  theoretii  approach  to  the  problem  of  i  lit,  creep  t.  i  tig  maneuvering 
i rent  n  vehicles  is  presented. 
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Chapter  / 


FUNDAMENTAL  PROBLEM  FORMULATIONS 


A.  I  ii  t  roduct  i  oil 

Tin*  purpose  of  t.liis  report,  is  to  present  the  results  of  work  don*- 
during  I9(>7  by  the  I  u  I'o  ruin  t  i  on  and  Control  l.uboratory  in  support  of 
Nihl!-\  System  Fvaluution  Studies.  The  present  report  covers  the  area 
of  optimal  guidance  and  control,  while  a  second  report  discusses  optimum 
ext  l ma lion*  ' 

The  basic  approach  taken  in  these  studies  is  to  treat  -he  intercept 
problem  using  the  methods  of  optimal  control  theory.  In  a  previous 
report  on  this  contract2,  the  intercept  problem  was  formulated  i:i  this 
framework.  There  it  was  shown  that,,  if  all  random  effects  are  explicitly 
taken  into  account,  i he  interception  of  a  ballistic  reentry  .vehicle  be¬ 
comes  a  special  case  of  the  combined  optimal  control  and  estimation 
problem''.  It  can  also  be  shown  that  the  problem  of  intercepting  a 

. . .  ug  reentry  vehicle  t.MHV)  is  a  special  case  of  a  stochastic 

differential  game'.  It  is  recognized  that  the  solution  of  the  general 
case  of  either  of  these  problems  is  not  computationally  feasible.  How- 
I'H'i,  in  lief.  2  it  was  pointed  out  that,  by  taking  advantage  of  the 
p.ii'iiiiilur  structure  of  tin-  intercept,  problem,  it  might  lie  possible  to 
i  edm  e  i  ompu  I  a  t.  i  on  a  I  t  et|ii  i  rcmeiit  s  l.o  t.he  point  where  a  realistic  solution 
(ould  be  obtained  on  a  computer  facility  with  the  capability  of  the  pro¬ 
posed  MKK-  \  computer  system.  This  report  summarizes  the  progress  that 
has  been  made  toward  this  end. 

In  the  next  section  of  this  chapter  the  problem  of  intercepting  a 
ballistic  target  is  considered.  First,  a  general  problem  formulation 
is  given.  Then,  a  solution  for  this  general  case  is  presented.  Next, 
the  nature. of  the  problem  before  the  AMM  has  been  launched  is  considered 
in  greater  detail;  a  computationally  feasible  solution  based  on  assump¬ 
tions  that  are  generally  met  in  practice  is  presented.  Finally,  the 


Si  I  Hill  i«in  niter  I  lie  AMM  Ims  lieen  I  iiiilielieil  is  extimineii;  a  realistic  yel 
<  .  I'|||I  II I  ,|!  i  nil  i!  I  I  \  feasible  siillltiiin  fur  lilts  ease  is  given. 

In  t  lie  iul  lowing  seel. ion  the  interception  of  an  MHV  is  briefly  con- 
s  i  ile  r  eil .  Emphasis  is  placed  on  the  differences  from  the  ballistic  case 
.tin!  how  these  can  be  explicitly  taken  into  account.  Ilaril-poi  lit  defense 
and  use  of  multiple  interceptors  are  both  discussed. 

In  the  final  section  of  this  chapter  the  remainder  of  the  report  is 
outlined.  Each  of  the  three  remaining  chapters  discusses  computer  pro- 
gra..s  that  have  been  written  for  solving  the  three  basic  problems  indi- 
i  a  t  eil  above  In  Chap.  If  a  program  bused  on  dynamic  programming  for  the 
case  of  intercepting  a  ballistic  target  when  the  AMM  has  not  been  launched 
is  discussed.  Chapter  III  presents  a  program  based  on  the  gradient  method 

lui  . . iso  of  intercepting  a  ballistic  target  when  the  AMM  has  already 

been  launched.  In  the  final  chapter,  a  game- t heoret i c  approach  to  the 
MHV  problem  is  described. 

II.  Eo  nn  ul  at  ion  of  the  Ballistic  Missile  Interception  Problem 

I .  General  Formulation 

The  problem  of  intercepting  a  ballistic  target  can  be  formulated  in 
tin-  following  manner.  First,  the  system  equations  for  the  ballistic 
i  .11  I'd  .hi-  wit  t  l  en  in  state- space  notation  as 

kf  ~  +  Bt  (l)* 

wile  re 

xT  -  State  vector  for  target 

!/■_  3  Random  forcing  function  vector  for  target  with 

known  probability  density  function,  /i(mT) 

t  -  Time 

£r  -  Vector  function  representing  the  external 
deterministic  forces  on  target 

(')  •--  d,dt  O 


Ml  ions  1 1  i  **•  tititni***  i  **«l  f  o  i  «>at  h  ■-t'liiiiii,  ;»,s  nn*  figuM’* 
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In  Met  f>  t  hr  Mr  veil  dimensional  s  I  at.  e  Vector  runt  ill  us  tlil'ee  ('.artesian 
|m  sil  I  on  enol-it  l  lint  es  (  i  ,  y  ,  mill  )  ,  tlil'ee  f.iirl  i's  i  mi  velocity  eon  ril  i  ii  n  t  es 
(  i  ,  \  ,  mill  • )  .  mnl  the  tat  in  of  air  density  to  ballistic  coefficient  (/■//(). 
■\s  discussed  in  He  f  5,  the  ballistic  coefficient  is  included  in  a  state 
\nriahle  because  it  cannot  be  del  e  nn  i  lied  <i  /nioti  lor  an  unknown  target.. 
The  function  fr  is  then  obtained  by  defining  drug  and  gravity  to  be  the 
only  external  forces,  on  the  target  and  by  taking  air  density  to  be  a 
locally  exponential  function  of  altitude.  Explicit  formulas  appear  in 
See.  Ill  of  lief.  5.  The  statistics  of  the  random  forcing  function  vector 
are  determined  by  approximating  the  magnitude  of  external  forces  not 
accounted  for  in  JT :  for  this  case  the  probability  density  function  for 
i.  .  is  gnussiuti  with  mean  zero  and  covariance  matrix  as  in  Sec.  IV  of 
lief.  5 

The  stale  vector  is  not  directly  known.  Instead,  a  vector  of  mtin- 
suieiiieuts  to  which  noise  has  been  added  is  available.  The  measurements 
are  a  function  of  some,  but  not  necessarily  all,  of  the  state  variables. 
The  measurement  equation  is 

zr  bf(xT)  +  vT  (2) 

wile  re 

:_r  -  Measurement  vector  for  target 

t  T  -  Measurement  noise  vector  for  target  with 
known  probability  density  function,  /» ( ) 

It  r  Vector  function  representing  the  measurements 
in  terms  of  the  state  variables  of  the  target 

In  He  I  f»  the  t  h  ree- d  i  mens  i  ona  I  measurement  vector  is  taken  to  be  three 
position  measurement s  in  radar  coordinates  (range  and  two  radar  angles). 
Ihe  (unction  /jr  is  determined  from  geometrical  considerations,  as  in 
Sec  Ill  of  Ref.  5.  The  measurement  noise  vector  for  this  case  is  based 
on  a  model  of  the  MSll  described  in  Ref.  6,  the  probability  density  func¬ 
tion  for  Cj.  is  gauss  ian  with  a  mean  determined  by  bias  terms  and  with 
the  covariance  matrix  shown  in  this  reference. 

The  time  at  which  computations  begin  is  denoted  as  fQ.  At  this  time 
an  (i  pi  tori  probability  density  function  of  the  target  state  vector. 

/■„  >  i  *  t  |, )  j  .  is  known  hi  Ref.  5  this  probability  density  function  is 
determined  by  the  track  initialization  procedure;  it.  is  gaussinn  with 
"ii  an  and  lovniiauic  mat  MX  determined  as  i  u  See.  IV  of  Ref.  f) 


Tin*  AMM  is  l  1*1*111  oil  in  a  .similar  manner. 


The  system  equations  are 


where 


*>  +  Ea 


CD 


yA  3  State  vector  for  AMM 

uA  =  Deterministic  control  vector  applied  to  AMM 

wA  =  Handom  forcing  function  vector  for  AMM  with 
known  probability  density  function  p(y>A) 

JC  =  Vector  function  representing  the  external 
forces  on  the  AMM. 


In  this  report  tlie  six-dimensional  state  vector  xA  contains  three  Carte¬ 
sian  position  coordinates  (x,y.  and  z)  und  three  Cartesian  velocity  coor¬ 
dinates  (r.y.  andz).  Because  the  aerodynamic  characteristics  of  the  AMM 
can  he  studied  in  advance,  ballistic  coefficient  need  not  be  a  stale  vari¬ 
able.  but  instead  can  be  a  tabulated  function  of  the  other  state  variables. 

The  deterministic  control  vector  contains  all  parameters  that  affect 
the  AMM  trajectory  and  that  can  be  commanded  directly.  For  an  AMM  with 
.lerodyu.imi  c  steering  only,  this  vector  contains  two  components  which  spec- 
i  f\  the  angle  of  attack  in  three-dimensional  space.  If  the  thrust  is  con¬ 
trolled,  it  also  is  added  to  the  control  vector;  otherwise,  it  is  a  tabu¬ 
lated  function  of  time.  Throughout  this  report,  it  is  assumed  that  the 
thrust  is  not  controlled;  however,  all  of  the  techniques  discussed  here 
can  easily  be  extended  to  the  more  general  case  where  the  thrust  is  con¬ 
trolled.  It  is  also  assumed  that  no  guidance  commands  are  given  during 
the  operation  of  first  stage.  The  control  vector  for  the  entire  duration 
of  first  stages  is  taken  to  be  two  parameters  that  determine  the  angle 
between  the  local  vertical  and  the  vector  from  the  missile  site  to  AMM 
position  at  the  end  of  first  stage.  For  any  chosen  angle  the  AMM  follows 
a  pre-programmed  angle-of- attack  sequence  to  reach  this  point. 

The  function  Xt  *s  determined  by  writing  expressions  for  the  external 
forces  on  the  .VMM,  namely  drag,  lift,  thrust,  and  gravity,  and  expressing 
them  in  terms  of  system  state  variables  and/or  tabulated  functions  of 
these  variables  and  time. 

It  is  possible  to  spccifv  a  probability  density  function  for  the 
random  forcing  function  vector,  u’4  .  However,  in  most  applications  it  is 
•*11 1  1  i  c i  out  to  set  r4  to  zero. 


4 


If  tics  i  rt'd ,  it  is  possible  to  stipulate  that  only  noisy  measurements 
(if  tin-  AMM  si  nl»*  vri'liir  ;i  ri‘  uvai  I  silt  I  •*.  In  this  fust-  tin*  mrnsii  lenient 


i*t|iiat  i  on  i  s 


who  re 


—A^—A  '  +  S.A 


=  Measurement  vector  for  AMM 

r4  =  Measurement  noise  vector  for  AMM  with  known 
probability  density  function,  p ( ) 

"  Vector  function  representing  the  measurements 
in  terms  of  the  state  variables  of  the  AMM. 

However,  in  many  practical  cases  it.  is  possible  to  assume  that  the  state 
v  r  i  tor  of  the  AMM  is  directly  known,  i.e.,  that  .*. 

At  l0  the  probability  density  function  of  the  stale  vector  of  the 
AMM,  /»„  !v4(  /  u )  1  ,  is  spitcified.  Again,  it  is  generally  assumed  that 
t_4(f0)  is  known.  If  the  missile  has  not  yet  been  launched,  then  £.^(t0) 
consists  of  position  at  the  launch  site  and  velocity  zero 

The  general  performance  criterion  includes  two  types  of  terms.  The 
first  term  is  an  integtal  reflecting  the  cost  incurred  during  the  flight 
of  the  AMM.  This  integral  criterion  is  written 


./)  =  /w[  l[xA(a)  ,uA(o).(j]do 


J  j  =  Cost  on  flight,  of  AMM 

/  -  Cost  per  unit  time  on  flight  of  AMM 
I l  -  AMM  launch  time 

t t  -  Intercept  time;  firing  time  of  AMM  warhead 
c  =  Dummy  variable  for  time 
E  =  Expected  value  operator. 

1 h e  expectation  must  be  taken  if  a  random  forcing  function  and/or  mea¬ 
surement  noise  affects  the  state  of  tlu*  AMM.  One  choice  of  this  criterion 
is  ,i  mi  it  t  mum  t  i  me- o  f  flight,  in  which  case  /  I  and  ./  (  |(  -  t  (  . 


*s h  V -1'i‘miJinl  cost  tinned  tiu  the  state  vectors  of  the  4 
iH  intercept  time.  Tins  t.erm  ran  lit*  expressed  as- 

;  ,  '  J2  -  F{\p[xA(,tf)  ,xTitf) ,  t  j]  I  (6) 


wltfl't* 


J.,  ^  Expected  Terminal  cost  of  tin  engagement 


,t p  “.Terminal  cost  function  of  an  engagement.. 

Tin-  expectation  must  be  taken  because  of  the  random  forcing  function  and 
measurement  noise  that  affect  the  target  and  possibly  also  the  AMM,  The 
terminal  cost  function  itself  generally  consists  of  two  types  of  terms. 

The  first  type  indicates  how  successful  the  intercept  is;  typical  choices 
include  kill  probability,  miss  distance  or  crossing  angle.  The  second  type 
reflects  where  the  intercept  occurs;  this  could  be  altitude  of  intercept  or 
some  other  function  of  battle  space  conserved. 

Constraints  are  imposed  on  the  ballistic  target  und  the  AMM  by  phys¬ 
ical  limitations,  such  as  maximum  acceleration,  maximum  aerodynamic  heatingy 
and  maximum  angle  of  attack,  as  well  as  by  operational  considerations  such 
as  fratricide  avoidance.  These  constraints  can  be  written 

X  y  £  Jfj* 


Ha  e  va 


where 


A’,.  =  Set  of  admissible  values  of  the  ballistic  target 
state  vector 

A'4  =  Set  of  admissible  values  of  the  AMM  state  vector 

U A  =  Set  of  admissible  values  of  the  AMM  deterministic 
control  vector 

The  sets  A'r  and  XA  can  vary  with  time,  while  the  set  UA  can  vary  with 
both  xA  and  time. 


The  problem  can  now  be  formulated  in  the  following  way: 


Given', 


(1)  Dynamic  equations  for  the  ballistic  target  and  AMM 


X_J  “  Xr(xT,  t)  Wy 

*a  =  LAlxA,uA.t)  *  y>A 


(2)  Measurement  nt  ions  for  tin*  ballistic  target  and  AMM 


i  9  I  \  A 


hr(xT,l)  +  vt 


‘,1  11 A  (  XA  '  '  >  4 

(3)  Statistics  of  the  random  forcing  functions,  the  measure¬ 
ment  noises,  and  the  a  priori  states  of  the  ballistic 
target  and  the  AMM 

p(vT),  p(Er)> 

p(nAh  p(eaK  /»oU4(to)J 

(4)  All  measurements  of  the  ballistic  target  and  AMM  up  to  the 
present  time,  t 

zT(t),  tn  '  t  ' 


<0  5  *5 


(5)  A  performance  criterion 

■I  -  .7  j  +  ■=  E  {j  l[xA(o)  ,uA(o)  ,v}ik  +  <{>[xA(t  f)  ,xT(t  f)  ,tf]} 

'  i 

(6)  Constraints  on  the  ballistic  target  and  the  AMM 

X7.  €  If  j. 

i/I  ^4 

Hi  e  {/, 


A  launch  time,  a  firing  lime,;  t  ,  and  a  control  policy,  uA(o)  , 

1 1  £  <-f-  such  that  the  performance  criterion  in  (5)  is  minimized,  where 

I  lie  expectation  is  conditioned  on  all  past  measurements  and  a  priori 
statistics,  and  where  the  constraints  in  (b),  the  dynamic  equations  in 
(1),  and  the  measurement  equations  in  (2)  are  all  satisfied. 

2  General  Solution 

The  problem  formulated  in  the  previous  section  is  a  combined  optimal 
(ontrol  am!  estimation  problem*.  A  general  solution  to  this  problem  can 
l.i-  obtained  1>\  using  dynamic  programming1,1'11.  For  simplicity  it  will  be 


a.'-. Mini imI  in  this  section  tliut  the  launch  time,  tt,  and  the  intercept  time 
nre  fixed.*  The  first  step  in  this  procedure  is  to  assume  that  all 
iih.scrvatious  and  controls  occur  at  discrete  times,  £0,  i0  +  At, 

t0  +  2A t,  ....  t0  +  K&t  -  tj.  For  simplicity  in  notation,  the  time  in¬ 
stants  tQ  +  feA (  are  then  abbreviated  as  k .  Also,  the  sequence  of  obser¬ 
vations  up  to  time  k  is  denoted  as 

Zk  -  {Z'T(k),z_A(k),z_  T(k  -  1),^  k  -  1),  Zj>  ( 0 ) ,  Zji  ( 0 )  } 

(8) 

The  sequence  of  past  controls  is  written  as 

Uh  =  {uA(k),vJk  -  1),  ...,  u^O)}  *  (9) 

The  differential  equations,  Eqs.  (1)  and  (1),  are  then  replaced  by 
the  following  difference  equations:' 

*7(  It  +1)  "  _jr(fe)  +  _[T  k)  ,  it]  At  +  j£T(  fe )  A  £ 

_^A(k  +1)  =  +  £a  [x4( k) ,  uA(  k) ,  fe]At  +  wA(.k)&t  .  (10) 

The  integral  in  Eq.  (5)  is  approximated  by  the  summation 


Z  l  [x4( k) ,uA( k) , fe] At 


where  kt  corresponds  to  launch  time,  i.e.,  fe(A t  -  tt  -  t0. 

Now.  the  minimum  cost  function  is  defined  for  all  k,  k  =  0,  1,  . 
and  for  all  possible  sequences  of  past  measurements  and  controls  as 


I(Z,;U._.;k)  =  Min 

U Atk) . u^(K) 


Ha 


lixtik)  ,u.(k)  ,k] At  +  </>[. 


where  the  expectation  is  over  x_T(k) ,  xA(k) ,  wT(j),  and  t »A(j) ,  j  =  k, 
k  +  1 ■  ■ ■ ■ ,  K;  vr( j)  and  vA(j),  j  =  k  +  1,  ...  K  condi t ioned  on  Zh 


Dm  d»»t rrm nat i on  of  f|  and  t j  requires  a  further  search  over  the  solutions  obtained  here.  The  nature 
i,f  M*.ir«  h  will  be  di*<us.sed  in  connection  with  specific  problems  in  the  next  two  sections. 
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.t  n  <1  /'j  _  | .  Using  the  approach  of  Hef.  3  and  assuming  that,  tin:  statistical 
variables  I  tr4  .  u^. ,  i>4  ,  i>}.  ,  t  A  (  0 )  .  ami  ^.(0)1  arc  unco  r  r«*  I  a  ted  with  each  other 
ami  with  themselves  at  different  lime  instants,*  the  basic  iterative 
equation  can  be  derived  as 


/<**;"»- >;*> 


Min 

Utk> 


l\xAik),uA(k),k]/\t 


t  ^ It \r(k)+  £r\xT(k) ,k]h l  +  u>j.(  Jr )  At ,  Jr  1  +  Vj.(fe  ’  l), 
hA{xA(k)  +  _£,[x4(fc),u4(Jr),J:]Af  +  wA(k)/\l,k)  +  ^(Jt  +  1), 

2*.  «,UU/*-,;*)]}  (13> 

where  the  expectation  is  now  over  x  ,.  (  J: '  ,  xA(k),  ui^ik),  wA(k),  Vj.(k  *  I), 
ami  r, (Jr  *  I),  again  conditioned  on  X t  and  j.  In  the  expectation, 
u.Ak),  uA(k),  iiT(k  *  I),  and  v  A  ( Jr  +  1)  are  independent  of  Xk  and  U*-j,  so 
that  the  unconditional  probability  density  functions  can  be  used.  However, 
the  probability  density  functions  for  xT(Jr)  and  x4(Jt)  depend  very  strongly 
on  these  quantities.  The  conditional  probability  density  functions  can 
be  computed  by  iterative  application  of  Bayes’  rule.  For  x_T(k)  the  rela- 
t  i  on  i  s 


+  \)/ZjU,l  I,} 


pllT(j  +  1  )/xT(j  +  1 ) ]  I  ^  +  1  )/xT( j)]p ix.T(j)/Zj ,Ujm] ljdxr( j) 


4  n/-‘r0  4  1)11  J  (/'f lr(i  4  l)Ar(/)1/»  xr(j)/XrUl,l)yixr{j)dxT(j  +  1) 


j  -  0,  1,  ...  Jr-1  ,  (  U) 

where 

J  ...  dxT( j) 

denotes  integration  over  all  possible  values  of  xT(j).  The  probability 
density  function  plz^j  +  1  )/*T0  +  1)1  is  found  from  Kq.  (2)  and 


|/i  i<  io  i  .in  lit*  ni.txnl  .it  t  lif  of  liffttumt  .nMilniii.il  si.n.-  >  .it  i  ah  I  *  >  t «,  ,u  t  mint 

i  t  >  i  it  ion 


knowledge  of  p[rr(j  *  1)).  The  probability  density  function 

/>  i  *  j.  ( j  +  1 )  /  xT(  j )  ]  is  determined  from  Eq.  (1)  and  knowledge  of  p[wr(j)]. 

The  p  mb  ub  i  I  i  ty  density  function  p  [  xT(  j )  !Z>  ,  V )  _  j  ]  is  known  1  rom  the  pre¬ 
vious  iteration.  In  order  to  begin  the  iterations,  the  u  priori  prob¬ 
ability  density  function,  />„[ *r( 0) ]  ,  is  used.  Formally, 

p[xT(0) / z0,u_ ^  =  p0!*r(0)]  (15) 

An  equation  similar  to  Eq.  (14)  can  be  written  to  determine  the 
conditional  probability  density  function  for  _x ^ ( fe ) .  The  only  difference 
is  that  plx-jij  +  1 )  /if  ( ./ )  1  is  replaced  hy  +  1  )  /  (  j )  ,uA  (  j )  ]  ,  which 

is  then  determined  from  Eq  (3)  and  knowledge  of  both  uA(  j)  and  p  ( _/')}. 

In  order  to  start  the  iterations  in  Eq.  (13), 

HZ(f;,/K_|;A')  =  VAil>\xT(K)  .,xa(K)  ,K})  (16) 

where  the  expectation  is  over  xT(K)  and  xA(K)  conditioned  on  ZK  and 

This  solution  is  extremely  difficult  to  implement  in  the  general 
case,  primarily  because  a  new  quantity  is  added  to  the  argument  of  the 
minimum  cost  function  every  time  a  measurement  is  received  or  a  control 
is  applied.  The  dimensionality  of  the  problem  thus  increases  with  time; 
and  even  a  problem  with  relatively  low- dimensional  state,  control,  and 
measurement  vectors  quickly  becomes  unmanageable. 

3.  Solution  Prior  to  Launch 

If  the  AMM  has  not  yet  been  launched,  then  the  launch  time  is  deter¬ 
mined  as  pari  of  this  procedure.  However,  in  an  actuul  engagement,  the 
launch  decision  must  be  based  to  some  extent  on  the  present  positions  of 
the  other  interceptors  and  targets.  In  this  section  it  will  be  assumed 
that  a  supeivisory  program,  referred  to  here  as  the  launch  doctrine, 
will  examine  the  solution  and  determine  whether  or  not  an  immediate  launch 
should  be  made.  Methods  for  enabling  the  launch  doctr»ne  to  make  the  most 
effective  use  of  the  information  obtained  by  this  solution  are  currently 
under  study  and  will  be  discussed  in  a  fdture  memorandum. 

Be  cause  of  the  specific  nature  of  the  intercept  problem,  it  is 
possible  to  utilize  the  results  of  Ref.  9  to  reduce  considerably  the 
amount  of  computational  effort  required  for  a  complete  solution.  In 
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i  li  i  s  ii'fiTi’iu'i'  it  is  shown  t  lint  tin*  |i  roredu  re  of  tin*  previous  sort  ion 
■  .in  hr  hrokrii  down  into  n  ntiiiib'  r  of  sim|)ln'  functions  with  no  loss  of 
sir  nr  ml  i  t  y . 

lln*  first,  slip  in  tht*  solution  method  ol  the  previous  section  is 
to  obtain  the  conditional  probability  density  functions  of  the  target 
stair  and  the  AMM  state  at  the  present  time.  In  the  case  of  the  AMM, 
because  it,  is  still  at  the  launch  site,  no  estimation  is  required.  The 
AMM  state  is  known  to  be,  with  probability  one, 


^(k) 


1 1  * 

*3. 

0 

0 

0 


(17) 


wlirtr  v.  ,  x.,s,  .t.js  are  the  Cartesian  coordinates  of  the  luunch  site  and 
where  the  zeros  indicate  that  the  AMM  has  zero  initial  velocity. 

With  the  state  ol  the  AMM  completely  determined,  estimation  of  the 
target  state  can  be  performed  independently.  Previous  work5  has  shown 
that  the  extended  Kalman  filter  is  quite  suitable  for  this  function. 

The  use  of  this  filter  assumes  that  the  conditional  probability  density 
1  unction  for  the  target  state  can  be  represented  as  a  multidimensional 
trail ssi  an  distribution;  formally, 

/>!  !.,.(*)  j  Zj  Nixr(k/k) .STlk/k)]  (18) 


wbrir  ,  j.t  k  k )  .  i  be  conditional  mean,  and  S.r(k/k),  the  conditional  rnvnr- 
i  am  r  ni.it  i  ix.  are  determined  recursively  as  in  Ref.  5.  A  new  calculation 
•  if  these  quant  i  ties  is  performed  each  time  a  new  measurement,  zT(k).  is 
in  rived  I -or  this  i  I'lisun  tin-  filter  is  pa  rt.  i  c  u  1  a  r  1  y  well -suited  for 
on  I ■ n r  est i ma t i on , 

Under  the  assumptions  of  Ref.  5,  the  recursive  equations  can  be 
w i i t t en  as  a  set  of  prediction  equations. 


X  (k 
"  7 

k  - 

1) 

=  £  ;-  ( k 

S,ik 

k  - 

1  ) 

-  <!>,  (  k 

\/k  -  1)  +  £rl'x  T(k  -  1/ k  -  \),k}ht  t 
I  )Sr(k  -  \/k  -  1  )<!>£(  k  -  I)  ♦  Qr(  k  -  I)  (19) 
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The  computation  of  optimal  stochastic  control  can  also  be  separated 
into  several  simpler  calculations.  First,  it  is  seen  that  the  control 
u4(fe)  has  no  effect  whatever  on  the  target  state.  Thus,  the  probability 
density  function  of  the  target  state  at  future  times  is  completely  spec¬ 
ified  by  knowing  the  probability  density  function  at  the  present  time 
and  the  target  dynamic  equations.  If  the  assumptions  of  Ref.  5  hold, 
then  the  probability  density  function  of  future  target  states  is  again 
gaussian10,  formally, 
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t>\  \rij).  y.k\  N\\Tij/k).sT(j/k)\ 


ttll  I'l'C 


The  mean  and  covariance  matrix  again  obey  recursive  relations10; 
for  the  assumptions  of  Ref.  5  these  equations  are 


xT(  j  *  1/fe)  °  x_T{j/k)  *  Ir[ir(  ]/k) 


ST(j  ♦  1/fc)  =  QT(j)ST(j/k)  QTT(j)  *  Q(j) 


wh  ere 


Work  is  currently  lining  performed  to  find  closed- form  expressions  for 
J  +  1/fc)  and  ST(  j  *  \/k)  for  certain  forms  of  /*;  analytic  prediction 
of  these  quantities  is  then  possible.  Results  of  this  study  will  be 
reported  in  a  future  memorandum. 


At  this  point  it  is  convenient  to  assume  that  there  is  no  uncer¬ 
tainty  associated  with  the  state  of  the  AMM.  Specifically,  it  is 
assumed  that  3  0,  =  xA,  and  vA  0.  Because  the  initial  state  of 

the  .-VMM  is  known,  because  the  AMM  dynamics  can  be  studied  in  advance, 
and  because  a  radar  beacon  can  be  mounted  on  the  AMM,  it  is  clear  that 
the  uncertainty  of  the  state  of  the  AMM  will  be  considerably  less  than 
that  associated  with  the  target  state.  Thus,  the  approximation  that  the 
AMM  >1  at  i-  is  known  exactly  is  justifiablt  from  an  engineering  point  of 


It  i  ,s  now  appropriate  to  reexamine  the  problem  to  he  solved.  The 
quantity  that  is  to  be  minimized  is  still  the  sum  of  ,  and  as  ex- 

pressed  in  Kq  (2).  However,  the  expectation  is  now  to  be  taken  only 
OV('r  i/(A).  furthermore,  since  launch  time  and  intercept  time  are  to 
be  chosen,  the  minimization  is  not  only  over  uA(k) ,  k  =  kt,  ....  K,  but 
also  over  kt  and  K  themselves.  The  problem  thus  becomes  that  of  find¬ 
ing  ./  *  ,  where 


■I  ~  Min 

<  k  )  t  k~k  , . 


A 

k=k, 


lixjik ),uA(k),k]&t  +  'f'ixjQi),  xa(K),K] 


The  equations  of  motion  for  the  AMM  are 

xA(k  *  l)  =  xA{k)  +  £A[xA(k)  fU^Ut)  ,k]&l  ,  (25) 

where  xA(k)  is  always  known  exactly.  The  constraints  on  the  AMM  are,  as 
be  fo  re 

1a  e 

aA  e  UA  .  (26) 

The  probability  density  function  of  the  target  state  for  any  intercept 
time.  K  is  known  as 

p[xT(K)/Zk]  =  N[xT(K/k) ,ST(K/k))  (27) 

where  iT(A'/fc)  and  ST(K/k)  are  determined  from  Eq.  (23). 

In  solving  for  J1 ,  it  is  useful  to  consider  the  following  subproblem: 

For  any  given  state  at  intercept,  xAUi) ,  find  the  control 
sequence,  ji^(fe),  k  =  fe(,  ...  K,  such  that  the  cost  on  the 
AMM, 

K 

J,  =  I  l[xA(k),uA(k),k]^t 
h=k  , 

is  minimized. 

Because  the  initial  AMM  state  is  known  to  consist  of  its  position 
at  the  launch  site  and  velocity  zero,  and  because  the  optimal  trajectory 
is  desired  for  all  possible  final  states,  it  is  appropriate  to  consider 
forward  dynamic  programming11  for  performing  the  optimization.  The  man 
features  of  this  procedure  are  summarized  in  Chap.  II  and  discussed  in 
more  detail  in  Ref.  11.  This  procedure  obtains  / * [x4(#0  ,K],  the  minimum 
cost  of  reaching  any  specified  terminal  state.  xA(K) ,  from  the  launch 
site,  as  well  as  the  function  u*  Cx^Of) ,  A]  .  which  specifies  the  correspond¬ 
ing  optimal  control  sequence. 

One  important  feature  of  forward  dynamic  programming  is  that,  if 
the  single-state  cost  function,  i,  and  the  dynamic  equations,  j_A , 
depend  only  on  the  time  that  has  elapsed  after  launch  and  not  on  abso¬ 
lute  time,  then  the  minimum  cost  function  and  the  optimal  control  sequence 
depend  only  on  the  state  and  elapsed  time,  and  not  on  the  actual  time. 

For  realistic  criteria  and  for  the  general  AMM  dynamic  model  used,  this 
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t ''  t  lie  rust*,  tints.  lln>  forward  tiyimtitif  |t  rog  ntmm  i  tig  solution  rati  be  written 
•  / '  <  <  ^  i ,  tlie  minimum  rust  of  reuchiitg  term 'tin!  state  mill  t  lie 

limit  ion  that  ilet  enn  i  nes  the  mrrespondi  ng  optimal  control  sequence. 

For  the  problem  defitied  above  the  computational  requirements  of  for¬ 
ward  dynamic  programming  are  such  that  it  is  not  feasible  to  obtain  a 
complete  solution  in  real  time.  l:\wever,  the  nature  of  the  solution  is 
such  that  the  function  I'{xA)  and  uA'( )  can  be  precomputed  off-line,  and 
the  solutions  recovered  in  real  time.  Thus,  even  though  the  specifica¬ 
tion  of  l'(xA)  and  may  be  quite  time-consuming,  the  recovery  of 

minimum  costs  and  corresponding  optimal  controls  can  easily  be  done  on¬ 
line.  Computer  programs  for  both  obtaining  the  solution  off-line  and 
recovering  it  on-line  are  discussed  in  detail  in  Chap.  II. 

Returning  to  the  original  problem,  the  mi  u  i  mi /.at  i  on  in  Eq.  (24)  can 
be"  rew  r  i  t  t  cn  as 

r  =  Min  (r(xA)*  E  {'i’{xAK)  ,xa,K])\  (28) 

K  •  J-A  \  -  T  (  *  *  / 

where  l'(yA)  lias  been  computed  off-line  by  forward  dynamic  programming. 

I  be  minimization  is  now  only  over  xA ,  the  terminal  state  of  the  AMM,  and 
k.  the  intercept  time.  The  optimal  control  sequence  corresponding  to  any 
given  xA  can  be  recovered  from  the  forward  dynamic  programming  results. 

•be  launch  time,  k,,  is  determined  froi..  K  and  this  control  sequence  If 
the  criterion  for  this  optimization  is  minimum  time,  then  K  -  fe,  is  given 
<liicct  ly;  otherwise,  it  can  he  computed  during  the  optimization  and 
s,"i'*il  along  with  /'(  i,)  and  »iJ(a4)  (see  Chap.  II  and  lief.  II  for  details). 

Ilic  op.  iini/ation  in  Eq  (211)  can  now  he  performed  by  n  direct  search, 
f  i  i  s  t  .  a  sc, tr<!i  is  made  iiii-i  a  finite  set  of  values  of  K,  the  intercept 

the  launch  doctrine  determines  which  times  to  consider,  generally, 
a  I  a i r I \  small  number  of  values.  For  a  given  value  of  K,  the  optimal 
•bone  ol  can  be  determined  by  another  direct  search.  First, 
l>'\r(h)  ZK  j  is  obtained.  Then,  based  on  this  distribution,  a  finite  set 
ol  values  of  xA  is  selected.  The  expected  value  of  </’  is  then  computed 
foi  each  value  ol  using  p  f  yT(  k)  /Z^  ]  .  The  value  I'(yA)  is  also  recov¬ 
ered;  lor  each  a4  the  launch  time,  kt,  is  determined  from  the  value  of 
k  -  ki  corresponding  to  l'(xi)  and  the  actual  value  of  AT .  If  this  value 
°  1  1 -s  at  or  alter  the  present,  time,  i.e.,  if  the  launch  time  is  phyx- 

i  i  .i  i  I  \  po-sihlc  thru  I  lie  sum  of  I'  and  the  expected  value  of  ■/>  is  formed. 
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if  not,  then  this  value  of  x_A  is  not  considered.  Finally,  the  sums  for 
admissible  vulues  of  xA  are  compared,  and  the  minimum  value  is  chosen. 

'I lie  minimum  values  for  each  K  are  next  compared,  and  the  smallest  value 
is  chosen  as  J'.  This  completes  the  direct  search  over  K  and  xA . 

The  value  of  J'  that  is  finally  obtained  is  then  relayed  to  the 
launch  doctrine.  If  the  corrt sponding  launch  tine,  kt,  is  the  present 
time,  then  a  decision  is  made  on  the  basis  of  the  entire  engagement 
whether  or  not  to  make  this  particular  launch.  If  the  launch  time  is 
in  the  future,  the  launch  doctrine  notes  this  and  decides  what  further 
computations,  if  any,  need  to  be  made  for  this  case.  If  the  launch  doc¬ 
trine  determines  that  an  immediate  launch  is  to  be  made,  then  the  entire 
tontrol  sequence,  uA(k) ,  k  =  kt,  ...  K,  is  recovered  and  transferred  to 
the  guidance  program.  This  control  sequence  is  then  followed  by  the  AMM 
until  the  guidance  program  determines  from  subsequent  measurements  that 
a  correction  is  required.  Procedures  for  performing  this  correction  are 
discussed  in  the  next  section. 

A  summary  of  this  procedure  appears  in  the  flow  chart  of  Fig.  1. 

First,  the  conditional  probability  density  function  of  the  target  state 
at  the  present  time  is  obtained  by  use  of  an  extended  Kalman  filter. 

Next,  this  conditional  probability  density  fun>.i.ion  for  future  times  is 
found  either  by  numerical  integration  or  analytic  prediction.  Then,  a 
direct  search  is  made  over  values  of  intercept  time  and  the  terminal  state 
of  the  AMM  in  order  to  find  the  minimum  value  of  the  performance  criterion 
in  Kq.  (24).  First,  the  minimum  cost  of  flying  the  AMM  from  the  launch 
site  to  the  terminal  state  is  recovered  from  results  that  have  been  pre- 

. puted  using  forward  dynamic  programming.  Next,  the  expected  value  of 

the  terminal  cost  function  for  the  AMM  terminal  state  and  the  intercept 
lime  is  evaluated,  the  expectation  is  performed  with  respect  to  the 
probability  density  function  of  the  predicted  target  state.  Finally,  the 
minimum  value  of  the  performance  criterion  is  presented  to  the  launch 
doctrine;  if  a  decision  is  made  to  launch,  the  sequence  of  guidance  com¬ 
mands  and  the  AMM  trajectory  are  recovered  from  the  forward  dynamic 
programming  results.  The  computational  requirements  of  this  .solution 
method  are  such  that  it  can  be  considered  for  use  in  a  real-time  on-line 
s  \  s  t  em.  The  use  of  an  extended  Kalman  filter  in  real  time,  which  is  con¬ 
sidered  in  Ref.  5,  is  currently  being  studied  further.  Analytic  pre¬ 
diction  is  clearly  feasible  in  real-time,  and  the  computational  require¬ 
ments  of  more  sophisticated  prediction  schemes  are  now  being  studied. 
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FIG.  1  FLOW  CHART  FOR  SOLUTION  PRIOR  TO  LAUNCH 
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TIh-  comp I ct  »•  dynamic  programming  solut-.iii  for  l'(vA)  n*qu i  res  it  consider¬ 
able  itmount  of  time;  however,  this  solution  cun  be  precomputed  off-line, 
.stored,  and  recovered  in  real  time.  The  direct  search  for  minimizing  J' , 
including  the  taking  of  the  expected  value  of  </*,  appears  to  be  feasible 
for  real-time  computation,  especially  if  the  launch  doctrine  and  the 
prediction  results  can  be  utilized  to  limit  the  search  to  a  reasonable 
number  of  values. 

4.  Solution  After  Launch 

Once  the  AMM  has  been  launched,  the  nature  of  the  problem  will  be 
changed  somewhat  from  that  of  the  previous  section.  In  the  first  place, 
the  launch  time  is  already  determined  and  cannot  be  changed.  In  the 
second  place,  the  determination  of  the  present  position  of  the  AMM  is  no 
longer  trivial.  Nevertheless,  the  solution  can  again  be  broken  down  into 
a  .'tombc!  <-i  simpler  tasks,  much  as  in  the  previous  section. 

The  probability  density  function  of  present  and  future  target  states 
can  be  handled  exactly  as  in  the  previous  section.  Thus,  the  target 
estimation  and  target  prediction  functions  need  not  be  modified  at  all. 

The  determination  of  the  present  state  of  the  AMM  requires  at  least 
some  computation.  It  is  generally  sufficient  to  assume  that  this  state 
can  be  perfectly  determined.  Howerei',  the  actual  computation  of  these 
values  requires  some  data  processing.  One  procedure  is  to  integrate  the 
W1M  dynamic  equations  forward  using  the  guidance  commands  that  have  been 
giic.  \n  alternative  procedure  is  to  utilize  radar  measurements  of  AMM 
position.  Generally,  a  simple  polynomial  filter  is  sufficiently  accurate 
to  obtain  the  complete  state,  although  an  extended  Kalman  filter  can  be 
used,  if  required 

When  the  AMM  state  has  been  determined,  the  optimization  problem 
that  remains  can  be  stated,  using  the  notation  of  previous  sections,  as 
fol lows. 

Ft  ml: 

K 

I  I  bA  (k),uA(k),k)  +  i p[xr(K), 

p 

(29) 
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In- re  k  i  .s  tin*  present*  timi*.  subject  In  t.  Ii  <*  AMM  tiyiiamir  <*i|iiut  i  mis 


\Aik  *  |)  xA{k)  *  lA\xA(k)  ,uA(k)  ,k\>\l  ,  (30) 

I  lie  rmi.st  raillt  S 


^  t  xA 

aA  e  UA  (31) 

a  ml  tin1  initial  condition 

xA(kp)  -  *<p)  (321 

where  is  tin?  present  AMM  state. 

Dec  tin  sc  the  present  AMM  state  varies  throughout  the  engagement.,  it 
is  not  appropriate  to  precompute  the  solution  using  the  forward  dynamic 
p i og rnmm i ng  algorithm  discussed  in  the  previous  section.  Also,  because 
the  statistics  of  the  predicted  target  trajectory  are  changing  during 
the  engagement,  a  solution  precomputed  by  backward  dynamic  programming 
i >  likewise  not  feasible.  Thus,  the  optimum  intercept  problem  in 
Kqs.  (2  9 )  -  (  32 )  must  be  solved  on-line  in  real  time. 

An  important  consideration  in  solving  this  problem  i3  the  avail¬ 
ability  of  the  optimal  guidance  sequence  and  the  corresponding  AMM  tra- 
j  ec  t  o  r\  for  the  problem  solved  at  the  previous  time  instant.  The  only 
d i I  I c rem  c  between  this  problem  and  the  present  problem  is  that  the 
p • e  sen  I  and  future  statislies  of  the  target  state  may  be  mod'  i ed 
slight  1%  by  the  additional  measurement  received  and  that  the  present 
VMM  state  is  changed.  If  these  differences  are  small,  it  is  to  be  ex¬ 
pelled  that  th<'  ...» 1  lit  i  on  to  the  previous  problem  is  an  excellent  approx¬ 
imation  to  that,  of  the  present  problem. 

One  method  for  utilizing  the  previous  solution  is  a  forward  dynamic 
programming  procedure  in  a  small  region  about  this  optimal  trajectory. 
Decause  this  region  covers  only  a  small  nn  ’ber  of  increments  in  each 
state  variable,  the  computational  requr rements  can  be  reduced  to  the 
point  where  an  on-line  solution  is  feasible.  This  approach  is  discussed 
in  C.liap.  II  and  considered  in  more  detail  in  Bef.  11. 
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Ail  alternative  approach,  which  has  received  considerable  attention 
in  the  literature,  is  a  gradient  method12-14.  Basically,  this  approach  is 
based  on  iterative  computations  using  a  linear  expansion  about  the  pre¬ 
vious  solution.  In  Chap.  Ill  a  first-order  gradient  method  is  discussed. 
The  method  is  explained  in  detail,  and  a  number  of  examples  are  worked. 

Based  on  the  results  of  this  chapter,  it  appears  that  on-line  com¬ 
putation  with  the  gradient  method  is  difficult  but  feasible.  If 
computations  are  repeated  each  time  a  new  measurement  is  received,  the 
changes  in  the  problem  will  likely  be  so  small  that  only  a  few 
iterations  are  required  to  obtain  the  new  solution.  Work  is  progressing 
on  second-order  gradient  methods  that  have  better  convergence  properties 
when  the  previous  solution  is  very  close  to  the  present  solution.  Also, 
if  these  changes  are  sufficiently  small,  it  may  be  possible  to  utilize 
a  given  solution  for  several  time  increments,  until  the  additional  mea- 
Mirements  change  the  solution  enough  to  justify  a  new  calculation. 
However,  this  possibility  has  not  been  studied  in  sufficient  detail  to 
draw  any  conclusions  at  this  time. 

Although  the  gradient  method  of  Chapter  III  assumes  a  fixed  final 
time  and  a  terminal  cost  function  that  is  a  function  of  only  the  AMM 
state,  the  extension  of  the  method  to  the  performance  criterion  of 
Kq.  (29)  has  been  worked  out.  The  additional  computations  required  are 
not  extensive.  The  modifications  to  the  procedure  of  Chap.  Ill  will  be 
discussed  in  a  later  memorandum. 

The  computational  procedure  for  this  case  is  summarized  in  the  flow 
chart  of  Fig.  2.  Target  estimation  and  prediction  are  performed  as  in 
the  previous  section.  The  present  AMM  state  is  obtained  as  using 

whatever  data  processing  methods  are  necessary  Based  on  these  data 
and  the  previous  solution,  which  has  been  stored,  a  new  solution  is  com¬ 
puted  by  the  gradient  procedure  The  optimal  guidance  command  for  the 
particular  time  is  transmitted  directly  to  the  AMM.  The  entire  solution 
is  retained,  both  for  use  the  next  time  the  gradient  procedure  is  applied 
and  for  communication  to  the  AMM  if  no  further  data  are  received. 
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FIG.  2  FLOW  CHART  FOR  SOLUTION  AFTER  LAUNCH 

(’.  Intercept  ion  of  an  MRV 

If  I  lie  l  a  rget  is  a  Maneuvering  reentry  vehicle  (MRV),  the  problem 
■  ■in  he  In  nun  I  at  ei)  as  in  See.  I),  the  only  difference  being  Unit,  the  MRV 
is  capable  of  altering  its  course  during  flight.  The  dynamic  equations 
<>f  the  target  'hits  must  be  modified  to  read 


t j  +  —T  (33) 

where  uT  is  the  vector  of  maneuvering  controls  and  where  _[T  now  includes 
the  maneuvering  forces. 

The  presence  of  controllable  maneuvering  forces  greatly  increases 
the  chances  of  the  target  escaping  interception  by  the  AMM.  The  degree 
to  which  these  maneuvers  are  assumed  to  be  evasive  determines  what  prob¬ 
lem  I  ii  null  I  at  i  on  is  .mp  rop  i  i  at  o.  Under  one  viewpoint,  il  is  assumed  that 


(lie  target  process  os  data  about  the  AMM  trajectory  and  then  takes  direct 
evasive  action.  Theprobiem  then  is  a  stochastic  differential  game.  Using 
the  notation  of  Sec.  B  and  Eq.  (33),  the  formulation  is  the  following. 

(.'(  ven: 


Dynamic  Equations  for 

the 

MRV 

and 

AMM 

xT  = 

Ir(* 

■T  ’  Et 

,  t) 

+  wT 

Ia^z 

■A  UA 

,t) 

+  Ea 

Measurement  Equations 

for 

the 

AMM 

II 

t- 

*•1 

h.jA 

( xT, 

t) 

+  P-TA 

La  a 

h.AA 

0 

+  P-AA 

(3)  Measurement  Equations  for  the  MRV 


— TT  = 

h  y  (  X  rp  1 

t) 

+  ETT 

II 

f— • 

tiAT^lA  ’ 

t) 

*  Hat 

(4)  Statistics  on  the  random  forcing  functions,  the 
measurement  noises,  and  the  a  priori  states  of 
the  MRV  and  AMM 


p(mTa)‘  >  P(ETA).  P^rP*  PoUr^o^ 

P(Eaa}'  P(Eat}’  P(-Zaa)>  P^Eat'*' 
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All  past  measurements  of 
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(7)  A  performance  criterion 


1  f 

J  E  i  lA\xA(o),uA(o),o]du 
'i 

.<  / 

-  J  lT[xT(o) ,uT(a) ,o]da 

‘  'o 

♦  '/’  I  *a  (  '/)  'X.r(  1  f)  ’  1  f] 

(8)  Constraints  on  the  MRV  and  AMM 

X  i*  c  Jf  j. 

U  J-  6  Up 

*A  L  XA 

Ha  €  11  a 


Ft  nil: 

A  launch  time,  £,,  an  intercept  time,  tj,  and  an  AMM  control  policy 
u4(  ')  ■  i  t  —  °  such  that  the  performance  criterion  in  (?)  is  mini¬ 

mized  over  all  MRV  control  policies,  ur(cr)  ,  t0  <  o  <  t  j,  where  the 
expectation  is  conditioned  on  all  past  measurements  and  a  priori  statis¬ 
tics  available  to  the  AMM,  and  where  the  constraints  in  (8),  the  dynamic 
equations  in  (1),  and  the  measurement  equations  in  (2)  are  all  satisfied. 

The  major  difference  between  this  problem  and  that  of  Sec.  B  is  that 
the  control  policy  selected  by  the  AMM  is  influenced  by  the  possible 
strategies  of  the  MRV.  Specifically,  the  AMM  is  attempting  to  minimize 
tin-  performance  criterion  based  on  the  measurements  it  receives,  while 
the  MRV  is  simultaneously  trying  to  maximize  this  criterion  based  on  its 
own  measure nent s.  The  criterion  thus  must  reflect  the  objectives  of  both 
sides.  It  may  then  be  appropriate  to  consider  in  the  terminal  cost  func¬ 
tion  the  damage  that  the  MRV  can  inflict  if  it  detonates  its  warhead  just 
before  being  intercepted.  In  addition,  there  may  be  constraints  that  the 
MR\  has  to  hit  a  particular  defended  area;  this  is  an  extremely  important 
consideration  in  hard-point  defense.  Finally,  it  may  be  desirable  to 
assign  a  cost  to  the  MRV  for  performing  certain  types  of  maneuvers;  a 
term  in  (5)  is  provided  for  this  purpose. 


Hi*  1  it  t  i  v  «•  I  y  I  i  1 1  1 1*  work  Ims  l»«*«*u  dn.ie  on  tin*  gnicriil  case  of  this 
|n  oh  I  cm.  '*  A  .summary  of  results  nhtuiucl  to  iluti*  will  u|»|i«*  ti  i*  in  it  Inter 
memo  randum. 

This  formulation  is  unrealistic  in  that  it  assumes  that  the  MHV  is 
capable  of  making  measurements  of  the  AMM  and  processing  this  data  on¬ 
board.  This  capability  appears  to  be  beyond  the  rapacity  of  MHV  threats 
in  the  foreseeable  future. 

A  simpler  formulation  that  has  received  considerable  attention  in 
the  literature  is  that  of  the  deterministic  differential  game15-17.  In 
this  problem  it  is  assumed  that  both  the  target  and  the  AMM  have  complete 
knowledge  of  each  other's  state.  Again,  the  assumption  that  the  MRV 
knows  the  state  of  the  AMM  is  unrealistic.  It  is  shown  in  Chapter  IV 
that  the  use  of  this  assumption  can  lead  to  results  that  are  unduly 
pessimistic  for  the  defense. 

A  problem  of  considerable  practical  interest  is  that  in  which  the 
.VMM  receives  measurements  of  the  MRV  state,  but  the  MRV  is  denied  any 
information  about  the  AMM  trajectory.  In  Chap.  IV  a  simplified  version 
of  this  case  is  studied  extensively  using  the  concepts  of  ordinary  game 
theory.  Although  the  problem  studied  is  still  far  from  a  realistic 
model  of  an  engagement,  the  results  are  the  first  obtained  using  this 
approach.  This  work  is  also  noteworthy  in  that  the  use  of  multiple 
.•VMM’s  to  intercept  a  single  MRV  is  allowed  and  the  defense  of  a  hardened 
site  is  taken  into  account. 

Further  work  is  currently  in  progress  on  the  last-mentioned  approach 
and  other  gaming  approaches  to  MRV  intercept18.  An  extensive  study  of  the 
problem  of  estimating  the  state  of  an  MRV  appears  in  Ref.  1.  A  report  on 
all  of  this  work  will  appear  midway  through  this  year. 

1)  Contents  of  the  Remainder  of  this  Report 

The  remainder  of  this  report  is  divided  into  three  chapters.  Each 
chapter  discusses  a  computer  program  for  performing  the  calculations 
required  in  one  of  the  problem  formulations  discussed  in  the  previous 
section.  Each  chapter  is  self-contained  for  the  benefit  of  a  reader 
interested  in  one  or  more,  but  not  necessarily  all  of  these  programs. 
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Cliiipt  it  It  discusses  u  i'li  rwii  rd  dyiinnii  <•  progrummi  iig  program  lor  use 
in  tin-  |i  if  I  an  mli  ral  nil  at  imi.  As  discussed  in  Sit.  B-.l,  this  prof?  ram  is 
to  ho  run  off-line,  ami  t  In*  results  are  to  bo  retrieved  in  roul  timo. 
Programs  for  tin*  on- lino  rooovory  are  also  doner  i  boil.  Tlio  basic  program 
is  written  for  general  AMM  dynamics.  There  is  also  eon. si  tlerabl  o  flexi¬ 
bility  in  the  choice  of  both  the  integral  cost  incurred  during  the  flight 
of  the  AMM  uiitl  the  terminal  cost.  Results  for  particular  cases  are  dis¬ 
missed  in  this  chapter. 

(diopter  III  discusses  a  first-order  gradient  method  program  for 
guidance  after  launch.  Again,  there  is  considerable  flexibility  in  the 
spot  ification  of  AMM  dynamics  and  the  performance  criterion.  Results 
for  a  number  of  specific  cases  are  presented. 

Finally,  C.liap.  IV  summarizes  the  results  that  have  been  obtained 
ii - 1  n a  game- theoretic  approach  for  intercepting  an  MRV  that  does  not 
reieite  information  about  the  AMM  trajectory.  A  comparison  of  these 
insults  with  those  for  a  di f fe rent i al - game  approuch  is  given.  Also, 
haul-point  defense  and  the  use  of  multiple  AMM’ s  for  the  interception  of 
a  .single  MRV  are  discussed. 
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APPLICATION  OF  DYNAMIC  PROGRAMMING 
TO  THE  PRELAUNCH  CALCULATION 


A.  I  ii  t  r  ml  lie  t  i  on 

In  ii  missile  defense  system  the  initial  trajectory  along  which  the 
antimissile  missile  (AMM)  is  launched  is  a  critical  factor  in  system 
performance.  As  discussed  in  the  previous  chapter,  the  determination  of 
these  trajectories  is  made  in  conjunction  with  the  launch  doctrine  as 
part  of  the  prelaunch  calculation.  This  chapter  describes  a  dynamic  pro- 
giamming  computational  procedure  that,  provides  an  extremely  usclul  tool 
tor  obtaining  these  trajectories.  A  computer  program  based  on  li;i«  pro¬ 
cedure  has  been  written.  This  program  can  be  used  either  to  precompute 
results  for  later  retrieval  in  an  on-line  system  or  else  to  evaluate  off¬ 
line  alternative  launch  doctrines.  1,2 

The  problem  considered  in  this  chapter  is  the  following: 

For  an  AMM  with  known  dynamics  located  at  a  given  launch 
site,  find  the  minimum- time  trajectory  from  the  launch 
site  t"  any  reachable  point  in  space  for  all  feasible 
values  of  the  ve lor i Ly- vector  angle. 

The  criterion  of  in i n i mum- 1 i me- 1 o- i n te rcopt  maximizes  the  altitude  of  inter- 
t  epl  ,  and  hence  provides  the  greatest  protect  ion  to  l, he  delended  area.  Fhe 
ang  I e- o I  - ' e I oc i t y  vcctoi  is  lonsidcred  because  ol  the  importance  ol  the 
iiossiiig  angle  —  i.e.  the  angle  at  intercept  between  the  velocity  vector  of 
the  AMM  and  the  velocity  vector  ol  the  reentry  vehicle  (RV) — in  the  ability 
of  the  AMM  to  compensate  for  errors  in  predicted  RV  position.  Such  errors 
iiir  always  present  because  of  uncertainty  in  the  estimates  of  present  RV 
position,  velocity,  and  dynamics. 

The  minimum  time  to  reach  all  possible  intercept  points  at  all  fea¬ 
sible  ve 1 oc i Ly- \ pc  tor  angles  is  required  in  order  to  determine,  on  the 
basis  of  statistics  of  predicted  RV  position,  whether  or  not  to  launch 
at  the  present  time  (Chap.  I).  If  the  decision  is  made,  to  launch,  the 
intcti  epl  point  is  .specified,  and  the  corresponding  AMM  lolit.l'ol  sequence 
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.iml  l  rtijortory  «•  mt-  be*  rel.r ioved.  Tin*  criterion  used  Id  make  this  decision 
i.s  .specified  us  part  of  the  Launch  doctrine.  It  should  be  emphasized  that 
tlie  smiK*  values  of  minimum  time  can  be  used  in  conjunction  with  any  launch 
doctrine  and  for  any  statistics  of  predicted  1W  position;  lienee,  these 
calculations  need  be  made  only  once  for  given  AMM  dynamics. 

Dynamic  programming  provide*,  an  effective  method  for  obtaining  the 
required  information.  Extremely  general  AMM  dynamics  can  be  handled;  and, 
it  desired,  a  performance  criterion  other  than  minimum  time  can  be  used. 
Constraints  introduce  no  difficulties,  and  they  actually  reduce  the  com¬ 
putational  burden  by  decreasing  the  size  of  the  region  that  must  be  searched. 
The  direct-search  procedure  guarantees  that  an  absolute  optimum  is  obtained. 
The  minimum-time  trajectories  to  all  reachable  intercept  points  for  all 
leasible  ve 1 oc i t y- vec t or  angles  are  determined  in  a  single  calculation,  and 
the  corresponding  control  sequences  can  be  recovered  from  these  results 
with  no  additional  computations.  This  latter  property  is  a  direct  conse¬ 
quence  of  the  use  of  forward  dynamic  programming3,  and  it  is  analogous  to 
i  In*  feedback  control  property  of  the  conventional  dynamic  programming  pro- 
icdtire4,5.  However,  it  should  be  noted  that  once  the  AMM  is  launched,  the 
icsults  of  this  program  do  not  necessarily  specify  the  minimum-time  tra¬ 
jectory  to  any  other  intercept  point  and  vel oc i ty- vec tor  angle.  If  later 
information  about  the  predicted  RV  position  dictates  a  change  in  these 
quantities,  then  the  use  of  the  gradient  method6-8,  or  some  other  technique 
based  on  perturbations  from  a  nominal  trajectory,  should  be  used. 

In  Sec.  B  of  this  chapter  the  basic  relations  of  dynamic  programming 
are  derived,  and  their  significance  is  discussed.  Section  C  describes 
forward  dynamic  programming  and  illustrates  its  application  to  a  simple 
example.  Section  1)  formulates  the  problem  of  finding  the  control  sequence 
that  reaches  a  specified  intercept  position  and  velocity  angle  while  mini¬ 
mizing  an  integral  performance  criterion  (see  Chap.  1).  The  performance 
criterion  considered  in  this  section  is  minimum  time,  but  other  cost  func¬ 
tions  can  be  utilized.  Section  E  describes  the  computational  procedure  used 
in  the  computer  program  which  obtains  the  desired  control  sequences.  In 
Sec.  F  representative  results  from  the  computer  program  are  presented. 


Ft  ml: 


The  control  sequence  u(  0) ,  u(K)  such  that  J  in  Kq.  (2)  is  mini¬ 

mi  /.oil  subject  to  the  system  equation  (1),  the  constraint  equations  (3)  and 
(-!■),  and  the  initial  condition  (5). 

2.  Derivation  of  the  Basic  Iterative  Functional  Equation 

The  dynamic  programming  solution  to  the  above  problem  is  obtained  by 
using  an  iterative  functional  equation  that  determines  the  optimal  control 
lor  any  admissible  state  at  any  stage.  This  equation  follows  immediately 
from  Bellman’s  principle  of  optimality. 1,3  However,  in  the  interest  of 
clarity,  a  direct  derivation  will  be  given,  and  the  principle  of  optimality 
will  then  be  interpreted  in  terms  of  this  equation. 

The  fiist  step  in  the  derivation  is  to  define  the  minimum  cost  func¬ 
tion  for  all  x_  e  A'  and  all  k,  k  =  0,  1  ...,/(,  as 

K 

I(x_,k)=  min  2  l  [x.(j )  ,u(j  ),  j]  ,  (6) 

. K  * 

who  re 

x_(k)  ~  £ 

The  summation  is  then  split  into  two  parts,  the  term  evaluated  for 
/  -  k,  and  the  summation  over  j  =  k  ♦  1  to  j  =  K.  The  minimization  is 
similarly  split  into  two  parts.  The  resulting  equation  is 

K 


--  min  min  U[x,u(k)  ,k]  *  2  l  tx(j  ) ,  u(j ) ,  j  ] }  .  (7) 

jMfc)  «(;)  j  =  k+  1 

; "  *+T,  .  .  ,,K 


Ihe  first  term  in  brackets  in  Kq.  (7)  is  not  affected  by  the  second  mini¬ 
mi  /at  ion.  Thus,  the  equation  becomes 

K 

/(*,&)  =  min  l[x.,u(k),k]  +  min  2  l  [x(j )  ,u(j)  ,j]  ,  (8) 

* )  #W)  ;=  *+ 1 

/=*+  1 . K 

1 1* second  term  in  brackets  in  Kq.  (8)  is  exactly  analogous  to  the  defini¬ 
tion  in  Kq.  (6),  where  the  argument  of  I  is  g  [*.,  u(k ) ,  k] ,  k  +  1.  Abbre¬ 
viating  u(k)  as  u.,  the  iterative  functional  equation  becomes 

/( min  {  /  (l,j» ,  k  )  +  /  fg(a  ,  u  ,  k  ) ,  k  +  111  .  (<)) 
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Tli  if  equation  is  a  mat  hemal  if  a  I  statement  of  Hi*  I  I  man  '  s  |>r  i  nr  i  pi  «•  of 
n|>l  i  in  a  I  i  ty .  It  states  that  the  minimum  cost  for  state  £.  at  stage  A  is 
found  by  choosing  the  control  that  minimizes  the  sum  of  the  cost  to  be 
paid  at  tin*  |>  i  i'scii  t  stage  and  the  minimum  cost  in  going  to  the  end  I  coin 
l  In*  stale  at  stage  A  +  I  which  results  from  applying  this  control.  The 
optimal  control  at  state  x  and  stage  A,  denoted  ns  n(x,A),  is  directly  yyy 
obtained  as  the  value  of  for  which  the  minimum  in  Eq.  (9)  is  attained. 

Since  Kq.  (9)  determines  /  (x ,  A)  and  u.(x,A)  in  terms  of  I  (£,  A  +  1), 

it  must  be  solved  backward  in  A.  As  a  terminal  boundary  condition, 

/(x,A')  =  min  [l(x.,.u,/0]  .  (10) 

a_ 

The  optimization  over  a  sequence  of  controls  is  thus  reduced  to  a  sequence 
of  opt  i  in  i  7a  t  i  ons  over  a  single  control  vector. 

3.  The  Standard  Computational  Algorithm 

In  the  standard  method  for  solving  l\q.  (9),  each  state  variable 
i,i  I,  2,  ...,  «,  is  quantized  to  Nf  levels  and  each  control  variable 

ii  t  ,  i  *•  1,  2,  ....  flt,  is  quantized  to  MJ  levels. 

Initially,  I(x_,K)  is  found  for  all  quantized  states  x  £  X  by  evaluating 
l(±,u,K)  for  each  quantized  control  u  e  and  choosing  the  minimum  value 
l>\  a  direct  comparison.  The  optimal  control,  u.(x.,/0,  is  the  value  of  u 
that  minimizes  l(x_,u,K).  In  many  problems  no  control  is  applied  at  the 
liual  stage  A';  m  this  case  /(x,A)  is  evaluated  directly  as  l(x_,K). 

Nc\t  ,  at  A  =  A’  -  1,  for  each  quantized  state  £  t  X,  each  quantized 
i  nut  nil  u  ■  i  is  applied,  and  the  next  state  g  is  computed.  The 

in  ini  emiit  lost  at  the  next  state,  /  f  g  (£,  u  ,  A ) ,  A’l  ,  is  found  by  interpellation 
using  the  values  of  /  ( _x_ ,  A  )  at  quantized  states.  The  quantity  /(£,£, A  -  1) 
is  computed  dii roily.  The  sums  of  these  quantities  for  each  quantized 
control  are  then  compared,  and  the  minimum  value  is  stored  as  l(x_,K  -  1). 

The  optimal  control,  £(£,A’  -  I),  is  stored  as  the  value  of  «.  lor  which 
tin*  in  i  n  i  mu  in  is  attained.  The  procedure  continues  in  this  manner,  with 
/  ( £ ,  A  )  and  ji(x,A)  being  computed  in  terms  of  I  (£,  A  +  1),  until  A  =  0  is 
i cached. 


For  wa ril  Dynamic  I’rogrnmmi ug 

tl  (lit*  in  i  ii  i  timiti  cost,  t'utiflion  is  redefined  to  be  I. lie  minimum  cost  to 
rrueli  a  given  state  ami  stage  from  the  i  ti  i  t.  i  a  I  state,  an  iterative  equa¬ 
tion  analogous  t.o  Kq.  (10)  can  lie  derived.  In  this  ease  the  calculations 
proceed  forward  in  k  rather  than  backward;  hence,  the  term  forward  dynamic 
piogMimm  i  ng  is  used  to  describe  the  computational  procedure  for  this  case. 

The  iterative  equation  is  derived  by  defining  /’ (*.,&)  as  the  minimum 
cost  to  reach  state  x  at  stage  it  from  the  initial  state.  Formally, 

*-  l 

l‘(x_,k)  =  min  1  l  [x. (;  )  ,u(j  )  ,j ]  (11) 

JU  (  0  )  ,_u(  1  )  .  .  .  M  k-  1  )  )  =  0 

where 

x  =  g  (x(it  -  1 )  ,u(k  -  ] ) ,  it  -  l] 

I  I  the  inverse  functional  to  g  is  defined  as  h,  so  that 

g(/i(x,u(it  -  1 ) ,  At  -  l],u(it  -  l),it  -  1 )  *•  x  ,  (12) 

then  t!**'  iterat'"**  equation  fcci  omes 

l'(x_,k)  =  min  (l{h[x ,u(k  -  1  ) ,  it  -  1  ]  ,  u (  fe  -  1 ) .  k  -  i} 

_u  (  ft-  I  I  ~  ~ 

+  I‘ {h.ix.,u(k  -  l),it  -  l],fc  -  1})  .  (13) 

•\s  a  boundary  condition  for  this  equation,  is  specified.  If, 

as  is  often  the  case,  the  initial  state  is  fixed  at  one  particular  value, 

/  (j.,0)  is  set  to  zero  lor  this  state  and  no  other  initial  state  is  con- 
side  red  udm  i  ss  l  i,  I  e . 

One  method  of  carrying  out  the  computations  in  Kq.  (13)  is  to  use  a 
procedure  exactly  analogous  to  that  of  the  previous  section.  In  this  case, 
at  a  particular  quantized  x_  and  k,  each  quantized  control  u(k  -  1)  is  ap¬ 
plied.  the  corresponding  previous  state  is  found  as  h[x_,u(k  -  1 ) ,  k  -  1  ] ; 

t  lie  minimum  cost  tor  this  state,  l(h,k  -  1),  is  found,  using  interpolation 
ii  necessary;  l[h,u{k  -  1 ) ,  k  -  l]  is  evaluated  directly;  and  the  optimal 
control  and  minimum  cost  at  x.,  A  are  obtained  by  comparing  the  quantities 
in  brackets  in  Kq.  (13)  for  all  quantized  controls  ji(fe  -  1).  This 


pimediiii-  ih  st  rat  i'll  t  forward,  hul  it,  does  require  compulation  of  the  in- 
\  >■  i  -I-  I  mill  i  Iin;i  I  /»;  this  cun  In-  di  Mil  lilt  if  g  is  u  general  non  I  i  near 
i  i  in  i  *  -  v  u  rv  i  n  g  f  iinrl  i  on  a  I  . 

A  p  i  ut  eiluiT  that  overcomes  this  difficulty  is  the  following:  At  euch 
quunt  i  /.fd  state  *(A  -  1)  where  /  1  [*.(  A  l),A  -  l]  has  just  been  computed, 
**«ii  h  quunt  i  zed  iidiu  i  ss  i  h  1  e  control  is  applied;  for  each  corresponding  next 
'tuie,  \  ( A* )  g!£(A  -  l),n.(A  -  1  ) ,  A  -  I),  a  check  in  made  to  see  if  it  has 

l-con  the  next  state  for  any  control  applied  at  previous  values  of  ,v(A  -  I) 
i I  it  has  not  previously  been  a  next  state,  then  the  quantity  in  brackets 
in  l\q.  (I'D  is  stored  as  the  tentative  minimum  cost  at  that  point;  if  it 
li.i'  been,  then  the  quantity  in  brackets  in  Kq.  (13)  is  compared  with  the 
tentative  minimum  cost  already  computed  at  that  point,  and,  i !'  it  is  less, 
t  h  i  -  ,o  i  ii  i  mum  cost  and  optimal  control  replace  the  values  stored  there.  If 
the  next  state  does  not  fall  exactly  at  a  quantized  state,  then  interpola¬ 
tion  is  lequircd.  This  procedure  continues  until  the  quantized  admissible 
(ontiols  have  been  applied  at  every  quantized  state  £(A  -  1).  The  tenta¬ 
tive  minimum  costs  and  optimal  controls  at  *»ach 

iU* )  *  g  m(A  -  I  ) ,  u(k  -  1 ) ,  A  -  l] 

uic  then  the  true  minimum  costs  and  optimal  controls  at  these  points. 

Since  this  procedure  is  to  be  used  to  solve  the  prelaunch  problem, 
n  simple  one -d i mens i ona 1  example  will  be  worked  out.  The  system  equation 
tor  t  Ii  i  s  prob  1  cm  i  s 


x  (A  +  1 ) 


flic  pci  I  ■  i  nil  a  in  I*  criterion  is 


x(k)  +  «(A) 


I  In*  i  ons  t  i  .i  i  n  t  s  a  re 


,/ 


•s  1  x 2  (  ./ )  +  «-(/)! 
/  0 


(14) 


(13) 


-I  1  «(A*)  <  I  (16) 

n  lid 

0  <  x{k)  <  2  (17) 


I  In*  initial  state  is  know  ii  t  o  be 


In  going  from  /'(*,  I)  Ln  !'(x,2)  u  c 
l  In*  (  a  leu  I  at  i  ons.  11  the  controls  are  fi 
tentative  minimum  costs  and  optimal  contr 
Asterisks  are  placed  beside  these  values 

When  the  controls  at  i(l)  =  1  are  ap 
i(-)  -  I  are  possible  next  states.  The  m 
t(l)  -  !  are  compared  with  the  values  air 
i  iiM’.',  less  cost  is  obtained  when  x(l)  = 

A'  '2  are  thus  »is  shown  in  Fig.  3. 

This  procedure  continues  until  minim 
been  (omputed  at  all  quantized  values  of  , 


Next,  suppose  that  tin1  liii ill  stilt. t*  can  li«*  iinywli«'r<*  in  tin*  region 
0x^2.  In  this  cast:  a  second  search  is  made  over  the  minimum  costs 
at  the  quantized  final  states,  and  the  final  state  is  taken  to  be  the 
mu-  for  which  the  minimum  cost  is  least.  In  this  example,  the  minimum 
.  ost  is  I'  -  7  iit  v  (  5 >  0.  The  optimal  trajectory  is  then  as  shown  in  Fig.  6. 

X 


Another  possibility  is  that  the  final  state  can  be  anywhere  in  the 
region  0  <  x  <  2 ,  but  there  exists  a  terminal  cost  function 

j[x(K),u{K)]  l[x(K),u(K),K]  .  (19) 

In  pnitirulnr,  suppose  that  (.lie  example  has  a  terminal  cost  of  the  form 

.*!*(.'>), u(r>)]  =  2.5U(5)  -  2]  2  .  (20) 

fable  I  shows  the  results.  When  the  terminal  cost  is  considered,  termina¬ 
tion  in  state  i  -  2  minimizes  the  cost  of  operating  the  process.  The  re- 

s u 1 t i n u  optimal  trajectory  is  shown  on  fig.  5. 

Ill  is  computational  piocedure  can  be  extended  to  more  complex  prob¬ 
lems.  If  the  next  state, 

1  =  g  lx  (fc  -  «) ,  «(  k  -  1 ) ,  It  -  1  ]  , 

does  not  occur  exactly  at  quantized  values,  some  interpolation  procedure 
..ns  i  lie  used.  The  simplest  technique  is  to  associate  the  next  state  with 
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the  n  capo  si,  quantized  stale  lor  purposes  of  cost, 
comparison.  The  optimal  cost  l(x,k)  will  be  the 
minimum  cost  chosen  from  all  states  lor  which  x_ 
is  the  nearest  quantized  value.  The  optimal  con¬ 
trol  and  minimum  cost  and  the  corresponding  state 
need  not  be  a  quantized  value.  This  eliminates 
the  need  for  interpolations  in  reconstructing 
optimal  trajectories  after  computations  have  oeen 
completed — an  especially  useful  property  if  the 
optimal  trajectories  are  physical  missile  trajectories. 

1).  Formulation  of  the  Minimum-Time  Problem 

The  purpose  of  the  solution  is  to  determine  the  minimum-time  trajec¬ 
tory  from  the  missile  launch  site  to  any  point  in  three-dimensional  space 
within  the  range  of  the  missile.  However,  it  is  apparent  that  minimum- 
lime  missile  trajectories  always  stay  in  a  plane  determined  by  the  inter¬ 
cept  point,  the  launch  point,  and  the  local  vertical  at  the  missile  site. 
Thus,  solution  of  the  problem  of  minimum-time  trajectories  to  all  points 
mi  a  vertical  plane  yields  the  solution  to  the  three-dimensional  problem. 
The  kinematic  equations  for  this  reduced  space  are  the  following: 

y  =  l/m[T  cos  (cp  +  a)  -  D  cos  (p  -  L  sin  <p] 

2  =  l/m[Tsin(<p  +  a)-Dsin<P  +  Lcos<p]-g  (21) 

wliei  e 

T  =  Tit  rust  of  AMM  (lbs) 

D  =  I)i  ag  along  flight  path  (lbs) 

ra  =  Mass  of  vehicle  (slugs) 

L  =  Control  force  applied  perpendicular  to 
flight  path  (lbs) 

>P  =  Flight  path  angle 

ft  =  Angle  of  attack 

g  =  Gravitational  acceleration  (ft/sec2) 


Table  I 

VALUES  OF  TE11MINAL  COST 
UNCTION  AND  HIK  TOTAL 
COST  OK  TKHMINATINC. 

IN  EACH  QUANTIZED 
STATE 


* 

J 

0 

TOTAl. 

COST 

0 

1  0 

0 

10 

1 

8 

2.5 

10.5 

•> 

7 

10 

17 

40 


Tho  quantity  L  is  the  commanded  lift  force  perpendicular  to  the 
flight  path.  It  can  be  expressed: 

L  --  i  pV2SCL  (23) 

win*  re 

-  l.i  ft  coefficient 

Thus,  once  a  desired  control  has  been  chosen,  the  desired  lift  coefficient 
is  computed  as  soon  as  the  other  parameters  in  the  equation  are  known. 

There  are  numerical  difficulties  because  of  the  nonlinearity  of  the 
parameters.  The  atmospheric  density  is  a  function  of  altitude  which  is 
best  determined  by  a  table-lookup  procedure.  The  lift  and  drag  coeffi¬ 
cients  are  functions  of  the  Mach  number  (which  in  turn  depends  on  the 
altitude  and  velocity)  end  the  angle  of  attack  required  to  implement  the 
commanded  lateral  acceleration.  All  of  these  relations  are  found  by  table 

lookup  and  then  assumed  to  be  constant  for  a  time  that  is  short  with  re¬ 
spect  to  the  rates  of  change  of  the  parameters.  However,  these  variations 
in  parameters  do  preclude  getting  closed- form  solutions  to  the  minimum-time 
p  robl em. 

The  state  of  the  missile  is  completely  known  if  the  position  and 
velocity  are  known.  Since  the  missile  is  restricted  to  motion  in  a  plane 
there  are  two  components  of  both  position  and  velocity — a  total  of  four 
state  variables.  If  the  assumption  is  made  that  acceleration  is  constant 
during  the  interval  (At)  in  which  the  missile  parameters  are  constant, 
the  following  integration  formula  is  used: 

y{  t  +  At)  =  y  ( t )  +  y ( f )At 

i( t  +  At )  =  z  ( t )  +  z  ( t )At  (24) 

and 

y ( f  +  At)  =  y(t)  +  [y(t)  +  y ( i  +  At)] 

t 

At 

z(t  +  At)  -  z(t)  +  —  [z(t)  +  z  ( t  +  At)] 

A  program  could  be  wri tten1  using  Eqs.  (21),  (22),  (23),  and  (24)  to 
del  ini'  the  state  space  and  the  missile  dynamics.  However,  this  would 
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Tin*  minimum  time  cost  criterion  is  a  Inaction  of  four  state  variables  — 
t  >vo  in  spec  i  Is  position  and  two  to  specify  velocity.  However,  the  rein- 
tivclv  low  maneuvering  capability  ol  the  AMM  with  respect  to  its  range 
suggests  another  approximation.  The  physical  characteristics  of  the  AMM 
reduce  the'  number  of  feasible  trajectories  that  reach  a  point  specified 
l>\  /I,  and  ■('  (the  flight  path  angle)  to  those  for  which  the  cost  criterion 
minimum  time  is  only  a  weak  function  of  velocity  magnitude.  Thus,  good 
n| prox imat ions  of  minimum  time  trajectories  can  be  obtained  by  ignoring 
the  dependence  on  velocity  magnitude  and  using  a  space  formed  by  quanti¬ 
zation  of  the  variables  h,  t°,  and  tp. 

The  use  of  tiiese  two  approximations  reduces  the  dynamic  programming 
formulation  with  five  degrees  of  freedom  (four  state  variables  and  the 
stage  variable  time)  to  one  with  three  degrees  of  freedom  (three  state 
\ai  i  allies — one  of  which  is  also  a  stage  variable).  A  problem  of  this 
.'l/.o  can  lie  solved  on  present-day  computers. 

The  AMM  maneuvers  aerodynatni ca l 1 y  and  any  change  in  direction  ton- 
'  ami's  cons  i  dc  i  ,ibl  e  hind  i<  energy  or  velocity.  Many  maneuvers  or  an  ex- 
l i cine  maneuver  will  result  in  so  much  lost  velocity  that  the  AMM  will 
clearly  not  he  on  a  minimum- time  trajectory.  For  long  flights  the  solution 
using  minimum  time  criterion  does  not  conserve  enough  velocity  in  the 
cails  stages  of  flight  to  yield  as  short  flight  times  as  using  a  criterion 
which  conserves  velocity  magnitude.  However,  lor  short  trajectories  in 
"Inch  the  velocity  lost  to  drag  is  not  significant,  the  mi n i mum- t ime 
ciiteriou  yields  shorter  llight  times  than  the  max i mum- ve lor i ty  criterion. 
Hus  ambiguitv  indicates  that  the  approximations  described  above  yield 
solutions  that  are  optimal  in  the  reduced  space,  but  not  necessarily  in 
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the  entire  posi t ion- vel oc i ty  state  space.  The  maximum  velocity  criterion 
yields  improved  results  in  those  cases  in  which  velocity  is  important, 
namely  for  long  flight  paths,  because  it  reintroduces  velocity  into  the 
problem.  Thus,  the  state  reduction  assumption  described  above  clearly 
does  lead  to  errors,  but  experience  has  shown  that  a  judicious  choice  of 
tost  criterion  keeps  this  error  well  within  tolerable  limits.  The  length 
of  I  lights  being  considered  determines  the  criterion  used  in  the  computer 
program  that  generates  minimum-time  paths. 

K.  The  Forward  Dynamic  Programming  Solution 


litis  section  describes  the  manner  in  which  forward  dynamic  program¬ 
ming  was  applied  to  the  solution  of  this  problem.  Equations  (21)  to  (24) 
are  the  basis  of  solution.  The  state  variables  ft  and  9  are  each  quantized 
into  a  set  of  fixed  discrete  values  that  serve  as  reference  points  in 
physical  space  for  the  trajectories  to  be  computed.  The  number  of  quanti- 
/.ation  levels  of  fp  at  each  point  (ft,  9)  in  physical  space  is  fixed,  but  the 
values  of  these  quantized  stales  are  defertni  tied  during  the  actual  applica¬ 
tion  ol  the  dynamic  programming  algorithm.  The  method  for  doing  this  is 
discussed  later.  The  control  options  available  are  also  limited  to  a 
discrete  set. 

lhe  second  veistcss  of  foiwurd  dynamic  programming  discussed  in  Sec.  C 
is  then  used  to  solve  this  problem.  A  number  of  mathematical  operations 
must  be  performed  at  each  point  in  state  space.  The  system  equations  (21) 
to  (.24)  are  used  in  the  program  to  determine  the  next  state,  because  the 
equations  of  motion  are  easier  to  use  in  Cartesian  coordinates,  and  con¬ 
version  from  one  coordinate  system  to  the  othf  is  easily  done.  At  each 
point  in  state  space  (ft,P,<p)  one  first  estimates  the  time  necessary  to 
leach  the*  quantization  level  in  ft  (at  t  +  At — call  it  ft')  at  the  velocity 
of  ft,  and  >p.  The  other  quantities  necessary  for  eva  luat  ion  o  f  Eqs,  (21) 
to  (21)  are  found  from  table- lookup  subroutines.  .Equation  (24)  is  then 
evaluated  so  that  v ( /  +  At)  and  z(t  +  At)  are  obtained.  Then 


«(f  +  At) 


'  y ( t  +  At)2  +  z  i  t  +  A  f ) 2 


is  computed.  If  this  is  not  close  enough  to  ft'  (within  3  feet),  At  is 
corrected  and  Eq.  (24)  is  reapplied  with  corrected  value  of  At.  Two  or 
th  iec  iterations  are  usually  sufficient. 


Tin*  next  step  is  ;i  si*  nr  cli  for  I  In*  proper  i|  limit  i  /.nl  ion  level  of 
lln-  ■'  win  rli  it. mi  1 1  s  Iitiii  I  lie  I  i' i  it  I  I  r  uj  I'r  I  ii  ry  is 


t  :ui 


: (t  +  At ) 
y( l  *  At) 


(26) 


Tin*  (juaiit. i  zuL i on  level  of  0  lor  which  (t4  -  6*' |  is  minimized  is  the  quan¬ 
tization  number  assigned  to  O' .  Finally,  one  must  determine  near  which 
(plant  i  /.at  i  on  level  of  ((>  the  trial  trajectory  terminates.  The  computer 
program  chooses  as  many  as  five  trajectories  for  each  ft',  O'.  These  tra¬ 
jectories  are  chosen  to  guarantee  a  minimum  spacing  A((>  between  each  of 
the  trajectories.  The  trajectory  that  best  meets  the  cost  criterion  is 
chosen  for  each  feasible  quantized  value  of  <p. 

F.  He suits 

The  program  described  in  the  previous  section  has  been  used  to  deter¬ 
mine  tin*  mi nimum- t ime  trajectories  from  the  launch  point  to  all  quantized 
points  in  ft  and  0  within  the  range  of  an  AMM.  It  was  implemented  in 
FOHTHAN  on  an  IBM  7090  computer.  When  trajectories  have  been  computed  for 
a  given  value  of  the  stage  variable  ft,  the  datu  for  the  preceding  value 
"f  ft  is  stored  on  magnetic  tape,  and  the  more  recent  set  of  results  is 
transferred  to  the  core  storage  that  the  older  set  had  occupied.  The  whole 
solutio  is  eventually  stored  on  magnetic  tape.  A  second  program  has  been 
written  which  recovers  trajectories  from  the  magnetic  tape.  The  terminal 
point  and  desired  terminal  angle  of  the  velocity  vector  are  entered  on 
punched  cards  and  the  trajectory  that  best  meets  the  specified  terminal 
conditions  and  the  corresponding  optimal -control  history  is  traced  from 
the  launch  site  to  terminal  state.  Thus,  the  computer  program  needs  to 
be  lun  only  once  for  a  given  set  of  missile  dynamics  and  constraints  to 
vield  a  complete  solution  for  all  terminal  conditions.  The  recovery  pro¬ 
gram  is  then  used  to  obtain  any  number  of  trajectories. 

\ 

A  typical  trajectory  obtained  by  the  program  is  shown  in  Fig.  8.  In 
this  case  the  terminal  position  was  chosen  to  be  y  =  131,000  ft., 

-  =  12,900  It.  The  terminal  velocity  angle  was  not  specified,  but  instead 
it  was  chosen  to  minimize  the  time  required  to  reach  the  point.  This  time 
was  21.96  sec,  corresponding  to  a  velocity  angle  of  -35.8°. 
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FIG.  8  REPRESENTATIVE  MiNIMUM“TIME  TRAJECTORY  TO  (y,z)  =  (131000,  12900) 

A  number  of  examples  have  been  run  using  a  realistic  set  of  aero¬ 
dynamic  parameters.  The  program  required  less  than  one  hour  to  compute 
mini  mum- time  trajectories  in  the  entire  operating  region  of  the  AMM.  The 
accuracy  of  the  solution  lias  been  checked  from  several  points  of  view. 

When  control  histories  from  this  program  wore  used  in  a  routine  that  into- 
giated  t ho  equations  of  motion  much  more  carefully,  the  deviations  in 
terminal  positions  were  0.2  percent,  of  the  trajectory  lengths.  This  in¬ 
dicates  that  the  assumptions  utilized  to  integrate  the  equations  of  motion 
do  not  invalidate  the  results. 

Studies  have  also  been  made  to  determine  the  deviation  of  trajectories 
obtained  i rom  the  true  optimum  trajectories.  A  second  optimization  scheme 
using  the  same  ideas  described  here  lias  been  implemented.  However,  instead 
ol  considering  a  large  region  as  the  solution  space,  this  technique  con¬ 
siders  only  a  region  surrounding  some  nominal  trajectory.  The  quantizations 
ol  the  state  variables,  stage  variables,  and  control  can  be  much  finer 
without  resulting  in  a  problem  too  large  to  solve.  Optimal  trajectories 
I i oni  this  formulation  approximate  the  continuous-space  optimal  trajectories 
mm  h  more  closely.  When  optimal  trajectories  from  the  original  formula* 

Lion  were  used  as  nominal  trajectories  in  this  scheme,  only  two-  or  three- 
percent  improvements  in  flight  time  were  noted.  Similar  improvements  were 
observed  when  a  gradient  procedure  (see  next  hapter)  was  used  to  generate 
minimum-time  trajectories.  These  results  are  sufficiently  accurate  for 
use  in  the  prelaunch  calculation  or  for  the  evaluation  of  alternative 
launch  doctrines. 
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ti. 

Iii  this  thuptrr,  tin*  application  of  a  dynamic  progrumtiii  ug  computational 
pco<  imIucc  to  the  prc  launch  calculation  in  a  missile  defense  syst.cm  has 

I . .  discussed.  This  procedure  determines  the  mi  n  i  mum- t  i  me  trajectory 

to  any  reachable  point  in  space  at  any  achievable  ve  loc i t y- vector  angle. 

This  i n formal i on  provides  a  basis  on  which  to  determine  whether  or  not 
to  launch  the  -\MM  at.  a  given  threat.  If  a  decision  t.o  launch  is  made, 
the  initial  control  sequence  and  corresponding  trajectory  for  the  AMM  are 
easily  obtained  from  these  data.  The  results  for  a  given  set  AMM  dynamics 
can  he  computed  in  a  single  run  of  a  computer  program  and  stored  for  use 
in  either  the  evaluation  of  alternative  launch  doctrines  or  in  on  on-line 
-\ s t cm. 

A  computer  program  based  on  this  procedure  has  been  written  and  tested 
lor  a  number  of  different  cases.  This  program  used  relatively  coarse  in- 
i  lement  • :  .  ,-s  and  a  simplified  state  description  of  the  AMM.  Analysis  of 
these  res-ilt.s,  by  meuns  of  a  gradient  procedure  and  a  dynamic  programming 
procedure  using  finer  increment  sizes,  showed  that  these  results  were 
sti  I  f  i  c  i  ent  I  y  accurate  for  this  application.  The  coniputut  i  onu  l  require¬ 
ments  ol  the  program  arc  very  reasonable,  considering  that  the  program 
needs  to  be  run  only  once  to  obtain  all  minimum-time  trajectories  needed 
in  the  prelaunch  calculation. 
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('.hap  tor  III 


APPLICATION  OF  GRADIENT  METHODS  TO  THE 
AMM  GUIOANCE  PROBLEM 


\.  I  II  I  l  iullic  t  i  (III 

Tins  chapter  describes  a  numerical  procedure  l'or  computing  angle- o f - 
attatk  commands  for  an  antimissile  missile  (AMM)  to  optimize  a  given  per- 
lonniince  criterion.  This  method  is  particularly  well  suited  to  optimizing 
.ilniiit  ,i  given  nominal  trajectory.  Thus,  it  can  he  applied  to  the  guidance 
ptoblcm  after  an  initial  trajectory  has  been  obtained  in  the  prelaunch 
i  a  1 1  u  I  at  i  on  (see  Sec.  B  of  Chap,  i  and  Bel'.  I).  The  performance  criterion 
■  oiisnh'i'eil  consists  of  terms  for  the  miss  distance  of  the  AMM  from  the 
l at  get  and  the  crossing  angle  of  the  AMM  relative  to  the  target  trajectory. 
In  addition,  the  AMM  angle  of  attack,  which  corresponds  to  the  control 
vaiiahle,  is  subject  to  certain  constraints.  In  See,  It  the  AMM  eijtialions 
o|  motion  are  described,  and  in  Sec.  (!  the  problem  is  formulated. 

This  optimization  problem  is  equivalent  to  a  classical  two-point 
boundai \ \ a  I  lie  problem.  The  gradient  method,  which  has  been  programmed, 
i  al<  ii  I  at  es  numerical  solutions  by  the  following  recursive  process: 

(1)  Forward  integration  of  the  equations  of  motion  for  the 
AMM  using  a  given  control  history 

(2)  Backward  integration  of  the  adjoint  equations,  which 
are  obtained  by  a  linearization  about  the  trajectory 
obta  i  tied  in  (  I  ) 

(.1)  Calculation  of  the  gradient  of  the  performance  criterion 
with  respect  to  the  control 

<11  Determination  of  a  new  control  history  that  enables  the 
procedure  to  iterate  toward  the  optimal  solution. 

llie  g  i  ad  i  e  a  l  method,  which  is  described  in  Sec.  I),  yields  optimal  guidance 
i  i iiiiiii a  u  d  s  (open-loop  control  bistories)  and  the  corresponding  optimal  tra- 

I  e<  t  oi  v  |  or  t  lie  AMM. 

Numerical  results  from  seveiul  illustrative  examples  are  presented 
III  Sec.  K. 
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MilSS  (slugs) 

tiruvi  tuti  on  it  1  acceleration  (ft/sec  *’) 


K 


1) 

pVl 

Drag  fon  e  ( ^(A/ , ft ).S  1  (lbs) 

(3) 

1. 

pV 2 

=  Lift  force  =  C.  (St,QL)S  — —  (lbs) 

L  2 

(4) 

ii  winch 

-  Deference  area  of  the  AMM  (ft*) 

P 

=  Atmospheric  density  (slugs/ft3) 

Drag  cue  1  f  i  e  i  ent.  ‘  ( * ()  (AO  +  (S1,d) 

1)  1 

CL(SI,  a) 

Li  ft  nil*  1"  f  ic  i  ent 

St 

=  Mach  number. 

Sinrc  slm  r  t.*  range  AMMs  are  being  considered,  t.lic  above  mode  I  assumes  it 
mini  nt  ,tt  i  njr  jut, |  I'lat  earth,  with  g  taken  t.o  be  it  eons  t.an  t  .  Also  p  is  as¬ 
sumed  to  be  a  function  of  the  altitude  z  only.  These  simplifying  assump¬ 
tions  can  be  removed  by  including  the  Coriolis  Bnd  centrifugal  forces 
u<  ting  upon  the  AMM,  and  by  expressing  />  and  g  as  complex  functions  of 
v  and  : .  I‘‘ur .  hermnre,  it  has  been  assumed  that  the  AMM  responds  inst.nn- 
. . usly  to  commands  in  u,  the  angle  of  attack. 

The  drag  coefficient  (■  lt  is  the  sum  of  Cfi  ,  the  zero- lift  drag 

O 

c  oe  I  I  i  c  i  eii  t  .  and  ('  f)  ,  the  induced  drag  coefficient.  The  drag  and  lift 

« 

. . .  I  i  c  i  cuts  tire  functions  .  the  Mach  number  SI,  which  depends  on  the 

altitude  ;  and  the  velocity  V,  and  the  angle  of  attack  tt  of  the  AMM. 

These  relationships  are  expressed  in  tabular  form.  In  addition,  the  Mach 
iiumbei  1/  and  the  at  mosplic  r  i  <  density  /'  are  given  in  tabular  form  as  func¬ 
tions  o  t  the  a  1 1.  i  tude  z  . 

The  term  L,  given  by  Ivj.  (41,  is  the  commanded  force  perpendicular  to 
the  I  light  path  ol  the  AMM.  The  desired  lift,  foree  for  maneuver  i  tig  is 
ui  hi  ted  by  choosing  the  control  a  appropriately.  Of  course,  CL  will  also 
affect  the  drag  force  D  as  given  by  Eq.  (3). 
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if  t|u.  four-dimensional  state  vector  of  the  AMM  is  del  mod  as 


H„.,i  the  differential  equations  (1)  and  (2)  can  be  rewritten  concisely  in 
stat*  variable  form  as  follows: 


2 

JL  v> 

i  (-c„s  a)  y] 


•  /(*,<*,  0 


Problem  Formulation 


The  AMM  guidance  problem  being  studied  can  be  formulated  as  follows. 

Find : 

a  control  history  a(t)  e  £l(x,t)  ior  tg  £  t  <  ,  (7) 

that  minimizes  the  deterministic  performance  criterion  (cost  function) 

■){x(tf)  1  (8) 

subject  to  the  constraint,  that,  the  state  x(t)  satisfies  the  AMM  equations 
dI  mot  ion  (Kq.  (0)1, 

X  =  f(x,0.,t) 

w  he  ■  <<  t  (the  initial  time),  l,  (the  final  time),  and  x(  I  )  are  given ,  and 

O  J 

represents  the  set  of  admissible  controls. 

The  necessary  conditions  for  an  extremal  solution  to  this  problem 
Iisim-  been  given  by  Hrcakwell  in  llel.  2.  Before  proceeding  further, 
ms  c ra  1  comments  about  tlii  s  problem  formulation  are  appropriate. 


For  the  eomput  e  r  prog  rain  that.  has  been  deve  I  op<>«l ,  |<t(f)| 

is  must. ra  i  lied  t.o  lie  less  than  or  r<|itul  to  45  or  that 
angle  of  attack  which  yields  a  lateral  acre  I eral i on 
maneuver  of  50  g  I  this  angle,  which  will  lie  denoted  hy  <l 
is  a  (unction  of  a  and  can  he  calculated  from  liq.  (4)1. 
Thus , 

|«(/)|  <  min  (45°, a) 

The  above  relationship  defines  the  set  of  admissible 
controls  n(x,/)  as  denoted  in  (7).  Of  course,  the 
computer  program  can  incorporate  constraints  on  the 
angle  of  attack  other  than  those  given  in  (9). 

Expressing  the  performance  criterion  J  in  terms  of 
the  final  state  x(t^),  as  in  (8),  entails  no  loss 
of  generality.  For  example,  if  an  integral 

*/ 

I,  Jjx,a,i)<h 

a 

is  to  !,e  minimized,  introduce  an  additional  state  x 
satisfying  the  differential  equation 

a  ---  /  '  (  a  ,  a ,  t ) 

o  J  u  1  1 

Note  t  hut  7  0  ( A- ,  a ,  /  )  is  the  integrand  of  the  above 
integral.  Then  ,)  is  the  quantity  to  be  mini- 

m  i  zed  with  A  „  ( f  () )  ~  0 . 

Terminal  cons  t  i  a  i  a  t.s  on  the  AMM  state  htive  not  been 
considered  explicitly  in  this  problem  formulation. 
However,  terminal  constraints  can  be  treated  as 
follows.  If  the  />•  dimensional  (where  p  <  4)  vector 

f  il lit*  t  i  on 

••/■!*<  t;)]  =  o 

represents  the  terminal  constrai  ut.s,  then  deli  tie  a 
new  performance  criterion 

'  i  at  ( f  / )  j  ~  .)  I  a:  ( /  f)  ]  +  /(|  |i/»[a  ( t  t )]  |  |  2  ,  K  >  0 

where  ./!«(/)  1  is  the  actual  per formanre  criterion 
t.o  be  mi  u  i  mi  zed.  Thus,  a  minimization  problem  sub¬ 
ject  to  terminal  constrai  ut.s  is  converted  to  another 
minimization  problem  without,  constraints.  With  A 
chosen  to  be  suitably  large,  it  is  reasonable'  to 
expel  I  that  l  lie  in  i  n  i  in  i  za  t  l  on  of  ./  '  1  a  (  /  ^  )  I  will  cause 
the  teiminal  constraint  in  Fo.  (10)  to  be  satisfied 


Iii  mi  .irhitrnry  degree  of  e  I  oseiiess  .  This  n  |i  p  r  o  si  c  li 
In  It  iimi  I  i  ng  If  rm  i  mi  I  constraints  is  generally  re  l‘e  r  red 
to  us  the  "method  of  penally  functions,"  and  is  de¬ 
scribed  in  more  detail  in  Hef.  3. 


(4)  It  is  not  necessary  for  the  AMM  initial  state  x(tu) 
to  be  completely  specified.  In  this  ease  the  un¬ 
specified  initial  states  can  be  determined  together 
with  a  (  I  )  to  in  i  ii  i  in  i  /«*  the  performance  criterion. 

For  example  in  the  launch  doctrine  problem,  it  is 
important  to  choose  the  optimal  direction  of  the 
initial  velocity  vector,  where  the  magnitude  of  the 
initial  velocity  is  fixed. 


15)  In  this  problem  formulation  it  has  been  assumed  that 
the  final  time  t  .  is  fixed.  This  assumption  was  made 
in  order  to  simplify  the  optimization  procedure  that 
lias  been  programmed  and  is  described  in  Sec.  I). 
Another  computer  program  is  being  developed  which 
will  solve  optimization  problems  with  ‘‘free"  final 


For  the  results  that  will  he  presented  in  Sec.  E,  the  performance 
i  riterion  is  u  combi  uni  i on  of  miss  distance  of  the  AMM  from  the  target 
.uni  crossing  angle  of  the  AMM  relative  to  the  target  trajectory: 


•I  \x(t  f)]  =  t.y(fy)  -  a]2  +  [z  (t  j)  -  b]  2  +  i v  tan 2  y(t  f) ,  u>>0  , 


</2(  ',) 


w  h  e  r  e 


« ,  b  =  Position  coordinates  of  the  target  (ft) 


d  Miss  distance  of  the  AMM  from  the  target  (ft) 


>T  ~  >  «  =  Crossing  angle  of  the  AMM  relative  to 
the  target  trajectory 


,»  T  =  Angle  of  the  target  trajectory  (above  the  hori¬ 
zontal  axis) 


y m  ~  Angle  of  the  AMM  trajectory  (above  the  horizontal 
axis). 


II  the  angle  yf  is  given  by 


.  ?*7^V*V*jV  7+7 


it  i.i  ii  In*  shown  I  li  a  I 


Lan  y 


y'’b  -  ZVa 

-V "  „  -  **’fc 


(13) 


1).  (irndicnt  Method 

The  optimization  problem  formulated  in  See,  C  ran  be  solved  by  using 
a  gradient  method  that  is  generally  referred  to  as  a  steepest-tloscent 
t  e<  h  it  i  •)  ii  e .  This  eoitipul  at  i  ona  1  procedure,  which  is  described  be.ow,  is 
very  similar  to  a  technique  described  by  Bryson  and  Denham  in  Ref.  4. 

The  steps  involved  in  the  gradient  method,  which  has  been  programmed, 
a  re  1  lie  f  o  1  1  ow  i  ng. 

Step  (1)  With  a  control  history  0! 1  ( (  ) ,  integrate  the  AMM 
equations  of  motion  (6)  from  the  initial  state 
*(/  ).  This  yields  a  trajectory  x  '  ( f  )  and  the 

cost'  ./'  -  ,)1. 

Step  (2)  Integral**  the  adjoint  equations 


\ 


I 


\F‘(t)}  7XI 


vv  i  t  h  boundary  conditions 


(14) 


(15) 


where  ■ 1  is  the  four-dimensional  vector  containing  the  adjoint  variables. 

It  should  he  noted  that  the  adjoint  equations  (14)  are  integrated  backward 
in  time  I  i  mu  I  lie  boundary  conditions  given  in  Kq .  (15).  The  4  x  4  matrix 

/'*'(/  )  is  obtained  by  linearizing  the  AMM  equations  of  motion  (6)  ubout  the 
trajectory  obtained  in  Step  (1);  i.e., 


l-'it) 


I'  i  mu  I'.q  s  .  (  5  )  and  (  0  ) 


dx 


it  <  an 


(  t  )  ,  a  *  ;  t  )  .  t 

lie  shown  I  hat 


this  inn  t  i  i  \ 


(16) 


is  given  by 
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(•sing  Kijs  .  (12)  am)  (13),  it  ran  hr  shown  that  the  boundary  conditions 
i  ti  Kq.  (15)  (in'  given  by 


2(.y  -  a) 


2 (z  -  b) 


27l(yt’l  "  z»'  «»,,)  -  2v(yv\  +  zv  nv  h) 


•  t.  v  1/  .  i 

o  a  6 


2rl(-y»avb  +  zv\)  -  2^(yu  ov +  zv\) 


*  ( t f) 


/  •  *  \  0 
i  (vr.  -  : i'  )  “ 

•  »•  <. 


'/  =  (yt'a  +  ^'6)2 


•S,'“P  (3)  Having  tli  idjoiet  variables  A'(t)  obtained  in  Stop  (2), 
the  gradient  of  the  performance  criterion  with  respect  to  the.  control  can 
be  calculated  as 


v./'(f)  -  f <;•(/)  1  V</ ) 


"heie  the  4  •*  1  matrix  G'(t)  is  obtained  by  linearizing  the  AMM  equations 
ol  motion  about  the  trajectory  obtained  in  Step  (1);  i.e.. 


C.'U)  ~ 


**(<)» ‘1*  (l),  c 


I  l  mil  I'.q.  ( h )  it  can  be  shown  that  t.li  i  s  matrix  is  given  by 


C'(l) 


»  1  (  <  ) ,  ■» '  t  t  )  i  r 


‘J.  fj  '.V1* 


I  ST\- 


)  -  ’■(tH"  “  *  r«  “). 


1  (  dC#,  d(.  L  \  (  z  y 

~  Zvl'^vil  *  T  l"  7  sin  “  *  7  c”s  “ 


Step  (4)  Using  the  gradient  obtained  in  Step  (3),  compute  a  new 
control  history 

a ,+ 1  ( f )  =  a'(t)  -  A^tC'U)]  V(t).  /X1  >  o  .  (22) 

The  step-size  y.'  is  determined  so  that  J‘+1  <  J 1 ,  where  the  cost  J 
results  from  the  control  «l+1(i).  The  quantity  /z'  is  also  chosen  to  in- 
(  reuse  the  convergence  rato  ol  tlie  gradient  method. 

Steps  (1)  through  (4)  are  repeated  until  this  iterative  procedure 
converges;  t.e.,  until  the  gradient  in  Eq.  (19)  tends  to  zero.  Hence, 
the  gradient  method  can  only  be  guaranteed  to  yield  a  relative  minimum 
for  the  cos t  J.  To  start  the  gradient  method,  an  initial  control  history 
■'(/)  must  be  provided.  In  general,  a  reasonable  control  history  a°{t) 
can  be  chosen  from  physical  considerations.  Indeed,  the  proper  choice 
of  i'(t)  can  ensure  that  this  iterative  procedure  will  converge  to  the 
a|,. solute  minimum  of  J.  Finally,  it  should  be  noted  that,  the  gradient 
method  described  above  yields  an  optimum  control  history, which  is  “open 
loop”  ( i .  e . ,  the  angle-ol'-attack  commands  are  not  based  on  the  AMM  state), 
and  the  corresponding  optimal  trajectory  ior  the  AMM. 


E.  He  s  u  1 1  s 

The  gradient  method,  which  is  described  in  Sec.  1),  has  been  programmed 
>n  A1.G0I.  and  mu  on  a  11-5500  computer.  To  solve,  the  AMM  equations  of 
motion  (6)  and  the  adjoint  equations  (14),  an  Adams -Mou 1  ton  (4-point)  in¬ 
tegration  routine,  with  a  constant  time  increment  ol'  0.1  second,  has  been 
employed.  The  program  has  been  used  to  compute  optimal  guidance  histories 
(and  the  resulting  optimal  trajectories)  for  the  performance  criterion 
given  in  Eq.  (12).  For  this  problem,  the  program  requires  approximately 
ioui  seconds  per  iteration. 


Severn  I  examples  have  been  run  using  a  set  of  aerodynamic  tables  lor 
the  drag  coefficient  CD  and  the  lift  coefficient  CL  that  are  representative 
of  a  short-range  AMM.  The  five  cases  described  in  this  report  are  summa¬ 
rized  in  Table  I,  which  gives  target  position,  target- t raj ectory  angle,  and 


Table  I 

TAHUKT  I'AltAMKTKHS  FOH  TIIK  OJMPUTKH  WINS 


cask  :> 

CASK  3 

CASK  4 

CASK  5 

at  ft) 

M  ft) 

45  x  I0:i 

50  x  103 

70  x  103 

40  x  103 

70  x  to3 

40  x  103 

70  x  103 

40  x  103 

70  x  It)3 

40  x  103 

yT 

-- 

-- 

90.0° 

b3. 5“ 

U* 

-- 

-- 

to9 
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weighting  factor  for  the  performance  criterion  J,  as  defined  in  Eq.  (12). 
In  Cases  I  and  2,  t.he  cost  ./  only  contains  a  term  for  miss  distance;  in 
Cases  3  and  4,  the  cost  .1  consists  of  terms  for  miss  distance  and  crossing 
angle,  with  a  constant  weighting  factor;  in  Case  5,  the  cost  J  consists  of 
terms  for  miss  distance  and  crossing  angle,  for  different  weighting  lactors. 
For  these  cases  the  following  system  parameters  have  been  used:. 


yn.) 

=  2.193  x  10* 

ft 

*<*.) 

=  8.895  x  103 

ft 

8.970  x  |03 

ft/sec 

,*(«.) 

=  2.290  x  ]03 

ft/ sec 

,  -  1 

0 

=  10.0  sec 

T 

«  0  lb 

m 

-=  32.78  slugs 

.S' 

1.355  I  t  - 

(23) 


The  initial  strte  x(to)  corresponds  to  the  state  ol  a  short-range  AMM 
immediately  after  engine  burnout;  after  t  ,  thrust  is  equal  to  zero  and 
mass  i  .s  equal  to  a  constant. 

The  results  generated  by  the  gradient-method  program  for  these  cases  are 
presented  iu  Figs.  2  through  (>.  In  Figs.  2(a),  3(a),  1(a),  and  5(a),  the  con- 
iiol  histories  foi  intermediate  iterations  are  shown;  between  successive 


■lot  *.  tlir  angle  of  alllick  is  I  j  in  i  I  •>()  l>y  I  In*  roust  mini  given  in  ( 0 )  ,  ill 
Figs.  2(b),  3(b),  4(h),  niiti  5(  1>) ,  I.  In;  trajectories  in  the  y-:  piano  for  inter- 
iiictiiaic  i  I  c  rat  i  tins  arc  given;  l.lio  dots  represent  one- second  intervals  in 
i  line.  ’! lie  curves  labeled  OPT  in  these  figures  correspond  to  the  solutions 
to  mhi eh  the  gradient  method  has  converged,  in  Figs  2(c),  3(c),  4(c),  and 
5  ( <  ).  the  performance  c  r  i  ter  i  on  J  i  s  plotted  versus  the  nt.mber  of  iterations. 
For  each  value  of  w  in  Case  5,  the  optimal  values  of  <i~  (  t  j)  and  tan2  )  ( t  f) 
i>ere  computed  using  the  gradient  method.  The  results  which  are  plotted 
in  Fig.  (),  demonstrate  th»*  trade-off  between  miss  distance  and  crossing 
angle  that  is  obtained  by  ...rying  the  weighting  factor  w.  The  slope  of 
(lit*  resulting  curve  in  Fig,  6  can  be  interpreted  as  being  a  measure  of 
this  trade-off.  Of  course,  this  slope  will  depend  on  the  geometry  of 
l  lie  specific  problem,  i  .  e .  ,  on  the  initial  AMM  state  and  the  target 
p.i  l  nine t  e  r  s . 

If  the  appropriate  identifications  are  made,  Cases  3  and  4  can  be 
(onsidered  as  applications  of  the  method  of  penalty  functions  (this  ap¬ 
proach  was  described  in  Sec.  C)  to  problems  where  the  terminal  constraint 
t  o  he  sat  i s f i ed  i  s 

>(i,)  =0  ,  (24) 

in  winch  •  is  the  crossing  angle  of  t.lie  AMM  relative  to  the  target  tra- 
irrtory,  and  the  actual  performance  criterion  to  be  minimized  is 

■I  -  f.v(f;)  -  <il-  +  f  z  ( t  r)  -  bP  »  dHtf')  ,  (25) 

in  which  (/  is  the  miss  distance  of  the  AMM  from  the  target.  For  w  -  10 9, 
the  giadicnt  method  converges  to  a  solution  for  which  the  terminal  con¬ 
straint  in  Kq.  (24)  is  nearly  satisfied  and  the  cost  in  Kq .  (25)  is  mini¬ 
mized.  The  results  for  Case  3  are 


■<'/) 

1.7  *  I0"1  rad 

'/(/,)  - 

122  ft; 

(2(> ) 

ami  I  nr  Case  1 

v  (/  f)  =  0.4  x  10"  ■'  rad 

>/(/,)=  16  it  (27) 
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For  all  .»r  these  compiit cr  runs,  the  gradient,  met.liod  was  initialized 
with  't'  (/)  0.  As  ran  lie  seen  I'rom  Figs.  2  through  fj,  this  rlioiee  lor 

(In'  initial  control  history  (and  the  resulting  initial  trajectory)  differs 
markedly  from  •  lie  optimal  solutions.  As  these  figures  indicate,  the 
gradi ent  method  i n i t i ol l y  has  a  rapid  convergence  rate  [see,  in  particular, 

Figs.  2(c),  3(c),  4  (c),  and  5(c)].  However,  as  shown  in  Figs.  4(c)  and  5(c),  the 
convergence  rate  of  the  gradient  method  is  relatively  slow  when  the  optimum 
is  being  approached. 

F.  (’.one  1  us i ons  and  Summary 

In  this  study  a  numerical  procedure  for  computing  optimal  guidance 
commands  for  an  AMM  to  optimize  a  given  performance  criterion  has  been 
described.  The  problem  formulation  is  presented  in  Sec.  C,  and  the 
gradient  method  that  has  been  programmed  is  detailed  in  Sec.  D. 

The  usefulness  of  this  computational  technique  is  demonstrated  in 
Sec.  K,  where  numerical  results  for  several  examples  are  described.  The 
gradi ent- method  program  is  a  valuable  tool  for  future  studies  in  the  area 
ol  AMM  guidance  and  control.  It  can  he  used  for  the  purpose  of  evaluating 
Mihopt  min  1  schemes  that  ace  proposed;  i . c . ,  it  can  provide  n  “yardstick” 
lor  per  I orniance  comparisons.  In  addition,  by  modifying  the  basic  gradient 
method,  it  may  be  possible  to  develop  on-line  computational  procedures  for 
generating  control  commands  for  the  AMM.  This  is  particularly  important 
in  obtaining  real-time  operation,  since  only  a  few  iterations  may  be 
i equ i red . 

The  gradient  method  is  particularly  well  suited  to  optimizing  about 
a  giien  nominal  trajectory.  Hence,  it.  can  lie  applied  to  the  control  prob¬ 
lem  after  an  initial  trajectory  has  been  obtained  at  prelaunch. 
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f "/in/*  / 1- f  IV 


APPLICATION  OF  A  GAMING  APPROACH  TO  MRV  INTERCEPTION 


\  I  nt  roduet  ion 

Successful  interception  of  ati  attacking  missile  depends  heavily  on 
the  performance  cha ract er i st i cx  of  t he  attacking  missile.  For  a  maneu¬ 
sering  reentry  vehicle  (MHV) ,  the  basic  characteristics  are  the  following: 


1 1 ) 

The  attacking  missile  is 
approaching  the  target. 

capable  of 

maneuvering  while 

(2) 

The  attacking  missile  is 
the  interceptor. 

not  cap  a  hit 

!■  of  observing 

Cl) 

The  interceptor,  on  the 

other  hand, 

is  capable  of 

observing  the  attacker  during  the  reentry  phase, 
using  highly  sophisticated  ground-based  radar 
systems. 


Under  these  assumptions,  the  question  of  optimal  behavior  becomes  extremely 
complicated  for  both  the  attacker  and  interceptor.  Houghly  speaking,  the 
attacker’s  problem  is  in  what  sense  is  a  maneuver  optimal  when  the  inter¬ 
ceptor's  position  is  unknown.  This  problem  induces  the  interceptor’s 
problem — to  predict  how  the  attacker  is  going  to  maneuver  and  how  to 
malieuv  er  to  i  nt  ercept  i  on 


One  possible  way  to  simplify  the  interceptor’s  problem  is  to  postu¬ 
late  that  the  attacker  will  optimize  a  given  function  of  his  trajectory. 
Then  the  optimal  attacking  trajectory  can  be  determined,  and  the  best 
<  mini crmensure  can  be  found.  This  approach,  which  is  essentially  equiva¬ 
lent  to  assuming  a  nonmaneuveri  tig  attacker,  is  highly  weighted  in  favor 
of  the  interceptor.  Unfortunately,  the  resulting  interception  policy 
depends  strongly  on  l  lie  postulated  attacker  objective  funrt  ion  and  may 
turn  out  to  be  extremely  bad  if  the  attacker  is  actually  maneuvering 
according  to  some  other  criterion.  An  intelligent  attacker,  thus,  will 
not  use  the  postulated  objective  function.  For  this  reason  this  approach 
is  not  considered  furthei 


79 


previous  page 

IS  BLANK 


Aim' her  possi  hi  o  way  to  simplify  the  interceptor's  problem  is  to 
relax  t  lie  sm  nml  e  ba  rar  t.e  r  i  s  t  i  e  ,  and  assume  that  the'  attaeker  i  v  capable 
of  observing  the  interceptor.  This  approach  leads  to  n  classical  dif¬ 
ferential  game  problem,  which  can  be  solved  by  existing  techniques. 
However,  the  solution  is  meaningless  for  the  attacker,  because  he  cannot 
implement  it.  for  the  interceptor,  however,  the  solution  is  a  valid 
pessimistic  answer  in  the  sense  that  it  provides  a  feasible  interception 
tontrol  policy  that  guarantees  the  interceptor  a  minimum  performance 
level  independent  of  the  attacker's  behavior.  By  ignoring  his  blindness 
tins  approach  favors  the  attacker,  and  as  u  result  the  guurunteed  minimum 
performance  may  be  overly  pessimistic.  Since  attempts  in  this  direction 
have  been  discouraging,  a  major  question  is  how  much  better  can  the  inter¬ 
ceptor  perform  by  taking  into  account  the  blindness  of  the  attacker. 

The  following  is  an  attempt  to  formulate  the  problem  of  MRV  inter¬ 
ception  with  the  basic  characteristics  above  and  to  study  the  result. 
Several  simplified  examples  are  solved,  both  under  this  formulation  and 
as  a  discretized  differential  game,  and  the  solution*  are  discussed.  The 
problem  of  generalizing  the  method  of  solution  used  for  these  examples 
is  under  study,  as  are  several  alternative  approaches. 

B  Problem  Formulation 

Consider  one  defended  area,  one  attacking  MRV,  and  a  defense  which 
may  have  one  or  more  AMM' s  for  interception.  The  dynamic  characteristics 
and  the  number  of  AMM’ s  are  known  to  both.  The  attacker  must  hit  the 
defended  point;  if  lie  is  intercepted  successfully,  he  suffers  a  positive 
loss  that  is  a  known  function  of  the  interception  point.  The  attacker 
knows  the  initial  positions  of  the  AMM’ s  and  his  own  position  at  all 
times,  whereas  the  interceptor  knows  not  only  the  AMM’ s  positions  at  all 
limes,  but  also  the  attacker’s  position,  if  the  attacker  is  within  a 
specified  region  near  the  defended  point.  This  region  is  also  known  to 
the  attacker.  There  is  assumed  to  be  no  noise  or  delay  in  observations 
or  calculations  and  no  reliability  problem. 

lhe  attacker’s  objective  is  to  minimize  his  loss,  and  the  intercep¬ 
tor’s  objective  is  to  maximize  the  same  loss;  however,,  the  choice  of 
whuli  side  is  considered  to  sufter  the  loss  is  arbitrary.  As  stated 
above,  the  attacker’s  loss  is,  for  example,  a  measure  of  how  short  he  is 
of  < ompletely  destroying  the  defended  point.  If,  instead,  the  direct 
damage  to  the  defended  point  were  considered  as  the  interceptor’s  loss, 
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tin-  nn  I  \  effect  would  lie  .1  switch  of  III  I II  i  111  i  /.«*  r  with  mil  X 1  III  I  /.IT.  when  l  In* 
attacker  is  chosen  to  he  the  minimi  zer,  and  the  damage  to  the  defended 
point  is  a  two-valued  function,  the  expected  value  of  the  objective 
becomes  the  probability  of  interception — a  convenient  measure  when  the 
effectiveness  of  an  interceptor  is  the  subject  of  interest.  The  assump¬ 
tion  implicit  in  equating  probability  of  non i nt ercep t i on  with  defended 
point  damage  is  that  the  attacker  will  destroy  the  defended  point  if  not 
intercepted.  This  assumption  is  valid  if  the  attacker  is  constrained  to 
use  trajectories  terminating  1 11  t  iie  defended  point 

Now.  assume  the  region  observable  to  the  interceptor  to  be  such  that 
once  the  attacker  gets  into  it,  he  cannot  get  out  without  missing  the 
defended  point  Since  the  interceptor  cannot  act  before  the  attacker 
enters  this  region,  this  is  the  only  portion  of  the  attacker  trajectory 
that  t.'  >f  interest.  Once  the  attacker  chooses  a  point  of  entry  to  this 
region  his  problem  is  to  minimize  and  the  interceptor’s  problem  is  to 
maximize,  from  this  point  on.  Once  this  problem  is  solved  separately 
lor  am  bound. ir\  point,  the  attacker  is  free  to  choose  a  best  entry  point. 

To  describe  the  problem  mathematically,  the  following  objects  must 
he  defined. 

(1)  Let  A  _  and  A  be  I  lie  state  spaces  of  at  t  acker  and 
interceptor.  1 espee t i v el y  (A,  is  the  direct  sum  of 
the  state  space  of  the  AMM* s  if  more  than  one  are 
available).  Let  ao(0)  and  *,(())  be  the  respective 
initial  states,  and  771  be  the  subset  of  Xa  describing 
the  defended  area. 

12)  Let  i/la  )  be  the  set  of  attacker  trajectories  /?a 
sIhi  ting  at  .x  a  ( 0 )  and  terminating  in  7’/?,  and  let 
/( ’  be  the  portion  of  the  trajectory  /?  executed 
tip  to  time  t.  Similarly,  let  {lii  )  be  the  set  of 
interceptor  trajectories  starting  at  .»  (()),  and 
/?,  be  the  portion  of  the  trajectory  It  t  executed  up 
to  time  I.  Note  that  there  is  no  terminating 
restriction  and  that  the  spares  are  different. 


It)  Let  1  i/<  (  1  (  .  I  )  )  be  the  set  o  f  eon  (  ro  1  s  available 
to  the  i  nt,  ei  eept  or  at  time  I  if  it.  is  1 11  state  1 
and  let  \VA  ,  ,  )1  he  a  set  of  functions 


r. •  in.  1 »,(/!,'), f  1 ) 

wliei  i-  1  (/!')  denotes  the  i  iitcrcepl.or  state1  at  time1 
/  on  the  t  raj  cct  «»ry  /!' 


(4)  Final  ly.  let  l‘(xa)  be;  tb*'  loss  to  the  attacker  i  1' 
i  lit  rrrept  i'll  at  state  \  tl . 

If  attacker  mid  interceptors  execute  trajectories  l\a  utitl  /I,  ,  respec¬ 
tively,  then  the  attacker  state  at  (first)  interception  may  be  calculated 
(if  not  intercepted  then  it  may  be  taken  to  be  on  the  defended  point). 
Denote  this  state  by  x (Ra.R,)-  If  the  attacker  executes  Ra ,  and  the 
interceptor  uses  a  function  U  . )  as  his  control  law,  then  a  trajec¬ 

tory  H  (Rg ,V t)  is  defined  by  the  interceptor’s  dynamics.  The  problem 
can  be  stated  now  as  follows: 

The  attackei  and  interceptor  choose  Ra  £  {Ra}  and 

UA . )  t  .))  in  order  to  minimize  and 

maximize,  respectively,  the  function 

Ux[Ra, «,(«.-!/,  )1  }  $  UR,.Ut)  ■ 

in  this  form  the  MKV  interception  problem  is  a  one-step,  two-player, 
zero-sum  game,  where  the  minimi zer’ s  strategies  arc  {Ra},  the  maximizer’s 
strategies  are  {(/,(.,.,.)},  and  the  outcome  function  is  L(Ra,U  t- )  .  A 
pair  of  strategies,  R*,U*(.,  ,  ),  is  considered  to  be  a  solution  if 

L(/?:,(/,)  .  URl.U  •)  <  L(Ra ,  V*) 

",  <  <",<  •.:•)}  *.  t  {«„> 

H-'nce,  by  choosing  the  corresponding  component  of  a  solution,  either 
placer,  separately,  guarantees  that  he  achieves  L(R*a,U*),  at  least,  and 
if  an\  player  makes  a  choice  that  is  not  a  component  of  a  solution,  then 
his  opponent  is  ab>e  to  make  a  choice  that  improves  the  outcome  for  him. 
It  is  well  known  that  a  necessary  and  sufficient  condition  for  the 
existence  of  a  solution  is  that 

min  ,  ,  max  .  L(R„,U.)  -  .  max  ,  min  ,  L(R„,U.)  , 

and  any  root  of  this  e<|iiat.iou  is  a  solution. 

It  should  be  noted  at  this  point  that  the  rttacker  trajectories, 
may  be  described  by  the  controls  producing  them,  which  may  be  con¬ 
sidered  as  functions  of  time  and  the  portion  of  the  trajectory  already 
impleted,  bi  t  not  of  the  interceptor’s  trajectory.  Thus,  the  sot  { /? u } 
may  be  equivalently  ■•‘placed  by  a  suitable  set  i  f  control  laws: 


<*'„(  ..>> .  vj.ni. i)  ■■  </»:>  -  «j 


thus  making  t  lit*  pro  l>l  cm  appear  slightly  more  symmetrical. 

The  prohlem  above  was  put  in  terms  of  trajectories  and  portions  of 
I  '’.ijn  lories,  in  order  to  provide  a  formulation  for  what,  is  intuitively 
the  problem  of  the  ultimate  best  each  side  can  do.  The  attacker  is  bound 
to  i  house  his  trajectory  in  advance,  and  all  lie  can  do  is  to  optimize  on 
this  choice,  and  the*  ultimate  best  the  interceptor  can  do  is  to  use, 

» i "c I \ .  all  the  information  about  past  trajectories.  However,  for  prac¬ 
tical  reasons,  it  may  be  appropriate  to  restrict  the  plavcrs  further  by 
icijiiiriiig  that  the  applied  controls  depend  oniy  on  the  present  states 
rather  than  the  entire  past  trajectories.  Whether  or  not  this  restric¬ 
tion  changes  the  game  ts  an  important  question,  and  while  it  seems  trivial 
that  it  makes  no  di  (Terence  i  f  this  coi  'traint  is  imposed  on  the  attacker, 
it  is  not  in  general  true  that  such  a  <  .istrainL  makes  no  difference  for 
the  interceptor.  With  such  rest  r  i  et,  i  on;  ,  a  new  game  may  he  considered 
in  a  single  slate  space,  A’#  ®  A'(  .  This  new  game  is  a  differential  game 
with  the  state-space  not  completely  observable  to  one  of  the  players. 

I  n fort unat el \ ,  no  general  method  of  solution  exists  for  this  kind  of  game. 

<.’■  Km  si  cure  of  Solutions 

In  general,  there  is  no  guarantee  that  a  solution  exists  to  the 
above  game  in  anv  of  the  versions  On  the  contrary,  we  would  expect  in¬ 
tuit  i v  »•  1 v  a  solution  not  to  exist,  because  existence  implies  the  possibll- 
it\  of  i  a  I  cti  1  at  i  ng  the  attacker's  trajectory  in  advance.  The  standard 
approach  in  such  situations  is  to  enlarge  the  given  sets  of  strategies 
bv  permitting  randomized  actions.  The  new  strategies,  which  are  called 
min'd  to  distinguish  them  from  the  old  or  pure  strategies,  are  probability 
distributions  on  the  sets  of  pure  strategies.  Ky  lousidering  the  dis 
tri but  ions  where  probability  on*  is  assigned  to  one  pure  strategy,  pure 
strategies  mav  be  considered  as  special  cases  of  mixed  strategies. 

I'lie  sets  of  pure  strategies  for  attacker  and  interceptor  are  thus 
icplaicd  bv  ■ ’ e  larger  sets  of  mixed  strategies  A  player's  choice  in 
the  game  be,  t  ■  the  selection  of  one  mixed  strategy  from  his  set.  The 
out  i  01111'  of  both  plavcrs’  choice  is  the  e  v  /ie  r  t  «•  il  value  of  the  probability 
distribution  thus  induced  on  the  nut  come- function  values.  In  practice 
the  plaver  then  will  make  a  select  ion  with  a  randomness  that  is  defined 
bv  the  in  1  x  tl  steatrgv.  and  execute  the  pure  strategy  that  results. 
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|  Ignoring  .so  mo  delicate  mathematical  d  i  f  f  i  on  I  l  i  os,  which  are  not  of 

i  i  utmost  here  and  which  can  ho  lakon  cure  of,  wo  may  assume  that  an  at 

least  approximate  mixed  solution  always  exists. 

i 

>  i)  A  Simp  1  i  f  i ed  Case 

i 

\  A  drastic  simplification  of  the  problem  is  achieved  by  quantizing 

the  state  spaces.  We  consider  the  following  case: 

The  state  spaces  are  discrete  sets  of  points,  and  the 
;  players  simultaneously  move  in  discrete  times.  For 

;  each  point  there  is  a  given  set  of  points  reachable 

:  in  one  move,  thus  defining  the  set  of  available  con- 

j  trols.  We  also  assume  that  all  the  attacker’s  per- 

i  mi tted  trajectories  (i.e.,  those  starting  at  the 

given  initial  point  and  terminating  in  the  target) 
are  uniformly  bounded  in  length. 

j  Under  these  assumptions,  the  sets  of  pure  strategies  for  each  player 

j  become  finite,  and  there  is  an  optimal  solution.  In  principle,  the 

solution  can  be  found  by  writing  down  the  resulting  game  matrix  and 
using  well-known  techniques.  However,  the  matrix  nay  be  found  to  be 
impractically  large — indeed,  even  for  the  extremely  simplified  examples 
presented  in  the  next  section  the  matrix  is  so  big  that  it  can  not  be 
written  down.  A  technique  that  has  been  developed  for  solving  these 
examples  may  be  described  in  the  following  general  terms.  The  attacker’s 
available  controls  are  described  on  his  state  space  as  a  directed  graph, 
t.e.,  i.  collection  of  arrows  describing  permitted  moves.  Suppose  the 
attacker  chooses  his  mixed  strategy  and  consider  a  minimal  cut  set 
separating  the  target  from  the  initial  condition,  i.e.,  a  collection  of 
arrows  that  when  removed  from  the  graph  disconnects  the  graph  between 
initial  condition  and  defended  point  Let  p  be  the  probability  that 
the  attacker  will  pass  through  the  ith  arrow  according  to  his  mixed 
strategy.  Select  any  state  that  the  interceptor  may  occupy  at  that  time, 
say  the  feth,  and  find  his  best  action,  given  that  he  knows  the  valuesp^ 
Let  the  outcome  be  l  k(p  f  .  For  the  attacker’s  mixed  strategy 

to  he  a  solution,  the  pt  must  satisfy 


min  max 

all  possible  values  k 
of  (pi  ....  p  t  ,  ...  ) 


4*(Pi*  —  p,»  — ) 
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,111(1  I  lie  value  III)  t  it  i  n  I'll  Is  a  Inni'i'  1)1 1 II II  (I  In  I  ll  «■  value  of  till'  gallic.  i  .  »■ .  . 
i  lie  mi  t  entiie  of  all  n|iliiiiiil  play  Tlie  required  m  i  n  i  ill  :i  x  run  then  lie  ob¬ 
tained  by  I  i  lien  r  |>  rug ramin  i  ng 

IK  I'iirefully  choosing  the  «|»|»  ro|)  r  t  at-  «*  cut  sets  in  appropri  ate  order, 
and  matching  the  probabilities  obtained  on  the  boundary  points,  the 
class  of  “worthwhile”  strategies  is  so  drastically  reduced  that  the 
sol nt i on( s)  can  be  found  The  matching  phase  requires  finding  the  srt 
of  it  I  I  solutions  of  u  linear  program;  a  method  for  doing  this  has  been 
dev e I  oped . 

h  Kxamptes 

For  the  examples,  consider  a  two-dimensional  interception  space  with 
a  single  defended  point.  The  state  of  any  involved  missile  (attacker  or 
nut  of  the  AMMs)  is  taken  to  be  its  position,  whi  -It  is  restricted  l.o  be 
at  anv  point  of  a  reel  align  I  a  i  ,  two-dimensional  grid.  Thus  the  state 
spaces  of  all  involved  missiles  will  be  described  on  the  same  grid.  The 
set  of  controls  available  to  any  missile  is  the  same  in  any  state, 
and  is  illustrated  b>  Fig.  I.  Further,  in  all  the  examples  the  attacker’s 
initial  state  is  four  units  above  the  defended  point.  Then  the  set  {/(„} 

11 1  ■«'  1  I  qualified  trsijei  tulles  is  I  mind  to  be'  as  described  by  the  solid 
graph  (lig.  2).  Ihe  "broken”  arrows  represent  permissible  controls  that 
•  annul  lie  used  because  they  cannot  be  continued  to  trajectories  termina¬ 
ting  in  the  defended  point. 

Ihe  initial  position  for  any  AMM  is  on  the  defended  point,  and  an 
interception  is  considered  to  occur  if  both  missiles  occupy  the  same 
position  simultaneously  or  move  along  the  same  segment  at  the  same  time, 
ilms.  an  AMM  move  is  described  on  the  same  graph,,  with  reversed  arrows. 
When  in  initial  position,  an  AMM  is  permitted  an  extra  “zero”  control, 
(.c..  to  remain  in  place,  thus  enabling  choice  of  the  best  time  to  launch. 
Ihe  loss  function  is  taken  to  be  two-valued  (0/1),  which  makes  the  prob- 
abilitv  of  interception  the  value  of  the  game.  The  examples  will  differ 
with  respei  t  to  where  the  loss  is  I,  and  with  respect,  to  the  number  of 
AMM’s  aval  I  able  to  the  defense. 


PRESENT  ATTACKER 
STATE 


STAVE  SPACES  GRID 


THE  SET  OF  AMM 

AVAILABLE 

CONTROLS 


/tv 


•THE  SET  OF  ATTACKERS 
AVAILABLE  CONTROLS 


PRESENT 
AMM  STATE 


1-iG.  1  AVAILABLE  CONTROLS  FOR  AMM  AND  ATTACKER 


FIG.  2  SET  OF  ATTACKER  TRAJECTORIES 
WHERE  INITIAL  STATE  IS  FOUR 
UNITS  ABOVE  TARGET 


E\ample  I  (Out*  AMM)  (see  Fig.  3.) 


ATTACKER  INITIAL  POSITION 


AMM  INITIAL  POSITION 


FIG.  3  EXAMPLE  Is  ONE  AMM  WHERE  INTERCEPTION 
MUST  OCCUR  AT  LEAST  TWO  UNITS  ABOVE 
GROUND 


Solution:  The  optimal  interception  probability  is  1/3  (value  of  the 
game) . 

(1  )  Interceptor  optimal  strategy 

a.  The  first  control  is  random  with  {.robability 
1/3  for  each  move 

b.  The  second  control  is  to  move  toward  the  side 
where  the  attacker's  first  control  lies. 

i  Attacker  optimal  strategy  is  not  unique 

a.  The  first  control  is  arbitrary 

b  T!i e  second  control  is  random  with  prob¬ 
ability  13  for  each  move 

\ I  I  the  solutions  to  the  problem  are  convex  combinations  (i.e., 
probability  distributions)  of  three  basic  mixed  solutions  that  may  be 
graphically  described  as  in  Fig.  4.  Observe  that  the  same  control  law 
for  the  interceptor  is  uso.d  in  the  three  solutions,  and  the  different 
trajectories  obtained  are  due  to  the  response  of  this  control  law  to 
different  behaviors  of  the  attacker.  This  control  law  guarantees  the 
interceptor  success  with  probability  at  least  1/3,  no  matter  what  the 
attacker  does.  Also  any  of  the  given  randomized  attacker’s  trajectories 
guarantees  him  passage  to  the  defended  point  with  probability  al  least 
2/3.  no  matter  what  the  interceptor  does. 
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FIG.  4  THREE  BASIC  MIXED  STRATEGIES  FOR  ATTACKER  (Example  1) 


Now,  for  comparison,  relax  the  basic  assumption  regarding  the  blind 
ness  of  the  attacker,  thus  turning  the  problem  into  a  discretized  differ 
ential  game.  The  optimal  strategy  for  both  sides  will  now  be  the  same 
control  law: 


(1)  If  both  missiles  are  on  the  same  vertical  line, 
take  each  possibility  with  probability  1/3. 

(2)  If  not ,  then 

a.  Attacker:  Turn  away  from  AMM  direction 

b .  Interceptor:  Does  not  matter  as  he  i s  lost  anyway. 

The  value  of  the  game  is  now  only  1/9,  i.e.,  the  interceptor  cannot 
succeed  with  probability  greater  than  1/9. 

Example  2  (One  AMMMsee  Fig.  5.1 


ATTACKER  INITIAL  POSITION 


(INTERCEPTION 
MUST  OCCUR 
IN  THIS  REGION) 


TA-Sltt-405 


FIG.  5  EXAMPLE  2:  ONE  AMM  WHERE  INTERCEPTION 
MUST  OCCUR  AT  LEAST  ONE  UNIT  ABOVE 
GROUND 


No  t  f  that  the  loss  function  differs  from  that  of  lix  ample  1. 

Solution:  Tin*  optimal  i  uterception  probability  is  (value  of  the 

game) . 

(1)  Interceptor  optimal  strategy 

a.  With  probability  1/7  go  straight  to  the  center, 
ignoring  what  the  attacker  is  doing 

b.  With  probability  6/7,  remain  in  the  initial 
position  for  2  units  of  time,  after  which 

the  control  depends  on  the  attacker’s  position 
as  shown  in  Fig.  6: 


T*-8l«t-40« 


FIG.  6  INTERCEPTOR  OPTIMAL  CONTROL  LAW  WHEN  INTERCEPTOR  REMAINS 
ON  GROUND  FOR  TWO  UNITS  OF  TIME 


(A)  Indicates  attacker  position.  The  solution  is  unique  for  the  interceptor. 

(2)  Attacker  optimal  strategy:  the  strategy  is  not  unique; 
all  optimal  strategies  at  convex  combinations  of  the 
six  basic  solutions  given  by  the  following  diagrams 
and  their  symmetric  transposes,  (see  Fig.  7.) 


tt-SIM-407 


FIG.  7  SIX  BASIC  MIXED  STRATEGIES  FOR  ATTACKER  (Example  2) 


For  comparison,  consider  again  the  differential  game  obtained  by 
permitting  the  attacker -to  observe  the  interceptor.  The  solution  is: 

(1) 


Interceptor’s  optimal  control  luw:  Wait  2  units  of 
time,  and  then  apply  controls  according  to  attucker’ s 
position,  as  before. 


(2) 


Attacker’s  optimal  control  law: 

a.  If  both  missiles  are  on  the  same  vertical  1  ine and 


(i)-distance  greater  than  2,  then  go  straight  down. 


( 1 1  )-distance  not  greater  than  2,  then  take  each 
possibility  with  probability  1/3. 


b.  If  not  on  same  vertical  line,  then  the  control 
is  determined  from  the  positions  in  an  obvious 
manner. 


Optimal  interception  probability  (i.e., value  of  the  game)  is  1/3. 
Example  3  (Two  AMM’s  located  on  defended  pointHsee  Fig.  8.) 


ATTACKER  INITIAL  POSITION 


T 


L ( X o )  =  I  (INTERCEPTION 
MUST  OCCUR 
IN  THIS  REGION) 


AMM  INITIAL  POSITION  t*-sim-4o. 

FIG.  8  EXAMPLE  3:  TWO  AMMs  LOCATED  ON  TARGET 
WHERE  INTERCEPTION  MUST  OCCUR  AT  LEAST 
ONE  UNIT  ABOVE  GROUND 

Solution:  Optimal  interception  probability  (value  of  game)  is  0.8. 

(1)  Interceptor  optimal  control  law:  With  probability 
0.4  one  AMM  is  sent  straight  to  the  center,  and 
the  other  AMM  waits  for  2  units  of  time  and  then 
applies  controls  as  in  Fxumplc  2.  With  probabil¬ 
ity  0.0,  both  AMM’s  wait  for  2  units  of  Lime,  and 
then  apply  the  controls  represented  by  Fig.  9: 


U*$IM-40« 


FIG.  9  INTERCEPTOR  OPTIMAL 
CONTROL  LAW  WHEN 
BOTH  REMAIN  ON  GROUND 
FOR  TWO  UNITS  OF  TIME 
(Example  3) 

Symmetric  controls  for  attackers  on  the  other  side.  For  attacker  in  the 
middle,  take  every  pair  of  different  controls  with  probability  1/3. 

(2)  Attacker  optimal  strategy:  Not  unique,  and  all 
solutions  are  convex  combinations  of  the  follow¬ 
ing  three  basic  solutions  (see  Fig.  10.) 


FIG.  10  THREE  BASIC  MIXED  STRATEGIES  FOR  ATTACKER  (Example  3) 


F  Co n cl  us i on s 

Although  conclusions  at  this  stage  of  development  must  be  viewed 
as  tentative,  there  ar  several  properties  of  the  optimal  strategies  in 
the  examples  treated  whose  consistent  appearance  warrants  remark. 

First,  in  all  examples  treated  there,  is  a  substantially  higher 
p  rob.th  i  1  i  t  v  of  interception  than  for  the  equivalent,  f  ti  1  I  -  i  n  fo  rum  t  i  on 
differential  game.  This  implies  the  possibility  of  a  significantly  more 
effect  l  vi-  de fense. 
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